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^ ■ Abstract. Wc construct a variant of Floer homology groups and prove a gluing formula 

for a variant of Donaldson invariants. As a corollary, the variant of Donaldson invariants is 
' non-trivial for connected sums of 4-manifolds which satisfy a condition for Donaldson invari- 

, ants. We also show a non-existence result of compact, spin 4-manifolds with boundary some 

homology 3-spheres and with certain intersection forms. 
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In this paper, we construct a variant of Floer homology groups and prove a gluing formula 
for a variant of Donaldson invariants introduced in [13] . 

As S. K. Donaldson showed in |3l IH 13 |6], the moduli spaces of instantons have crucial 
information about the topology of closed, oriented, smooth 4-manifolds X. Mainly there are 
two methods to draw out the information from the moduli spaces. First one is to describe the 
■rj- structures of the singular points and the ends of the moduli spaces |3l H]. The description gives 
^ strong restrictions to the realization of unimodular quadratic forms as the intersection form of a 
Q . smooth 4-manifold. Second one is to count the number of points of 0-dimensional moduli spaces 
or to integrate cohomology classes over moduli spaces [SIE]- This gives differential-topological 
invariants of 4-manifolds which distinguish different smooth structures on the same topological 
4-manifold. 

It is hard to compute Donaldson invariants in general. However Floer theory gives us a 
way to compute Donaldson invariants \E'x when X has a decomposition X = Xq Uy Xi for 
^ ] some compact 4-manifolds Xq and Xi with boundary Y and Y. Here y is a closed, oriented 3- 
' manifold and Y isY with opposite orientation. A. Floer introduced instanton homology groups 
HF^iY) for homology 3-spheres Y in [TU]. Floer's groups allow us to generalize Donaldson 
invariants for compact, oriented 4-manifolds Xq whose boundaries are homology 3-spheres 
Y. The relative invariant Xo is an element of HF^(Y), which is defined by counting the 
number of points of the moduli spaces of instantons over Xq = Xq U {Y x M>o) or integrating 
cohomology classes of moduli spaces . Assume that a closed 4-manifold X has a decomposition 
X = Xq Uy Xi for some homology 3-sphere Y and two compact 4-manifolds Xq,Xi with 
boundary Y, Y and with 6"^(Ao), b^{Xi) > 1. Here stands for the dimension of a maximal 
positive subspace of the intersection form on H2{X). Then we have the relative invariants 
^'xo e HF^{Y), ^'xi e HF^{Y). There is a natural pairing 

< , >:HF,{Y)®HF,{Y)^Q, 

and we have a relation 

(1) =< ^Xo,^Xi > • 

Floer homology groups also enable us to generalize the results on the non-existence of closed 
4-manifolds with 6+ = and with non-standard intersection forms. Donaldson [7] showed that 

1 



2 



HIROFUMI SASAHIRA 



if HFiiY) = then there is a restriction to the reahzation of unimodular quadratic forms as 
the intersection form of a 4-manifold Xq with boundary Y and with b'^ = 0. The proof involves 
the description of the singular points and the ends of the moduli spaces of instantons over Xq 
as in the case of closed 4-manifolds. 

On the other hand, there are some variants of Donaldson invariants. R. Fintushel and R. 
Stern defined a variant of Donaldson invariants for closed, simply connected, spin 4-manifolds in 
[9j. They used a 2-torsion cohomology class ui G H^{Mp] Z2) of moduli spaces Mp which was 
originally defined by Donaldson in [4J. In [Ij, S. Akbulut, T. Mrowka and Y. Ruan extended the 
construction of Ui to non-spin 4-manifolds and they showed a universal constraint of Donaldson 
invariants for non-spin 4-manifolds. Using Ui for non-spin 4-manifolds, the author [13] defined 
a variant of Donaldson invariants for non-spin 4-manifolds , which can be regarded as 
an extension of Fintushel-Stern's invariant to non-spin 4-manifolds. A remarkable feature of 
these variants is that they are non-trivial for connected sums of the form X = Y^S'^ x S'^ in 
general. 

In [12], it was announced (without proof) that K. Fukaya, M. Furuta and H. Ohta showed a 
non-existence result of compact, spin 4-manifolds with fe"*" = 1 and with some intersection form 
when their boundaries are diffeomorphic to some homology 3-spheres. This is a generalization 
of a result in [1]. In [12], Furuta explained that a variant of Floer homology groups and an 
extension of to compact, spin 4-manifolds with boundary a homology 3- sphere are used in 
the proof. 

The main theme of this paper is to construct a variant of Floer homology groups and to 
extend the relation ([1]) to \I'^^ . As a corollary, we obtain a non- vanishing result for connected 
sums of some 4-manifolds. Our construction can be applied to not only the case when 4- 
manifolds are spin but also the case when 4-manifold are non-spin. Moreover we will write 
down a proof of Fukaya-Furuta-Ohta's non-existence result using the variant of Floer homology 
groups. In the proof, we make use of the structure of the ends of some moduli spaces as in [3] . 

In [13] the author showed that the variant of Donaldson invariants is non-trivial for 2CP^7^C^. 
As we will explain in Section [3l this calculation is suggesting that we need to take into account 
the following aspects in the construction of the variant of Floer homology groups: 

• The trivial fiat connection on Y should play an important role in the gluing formula 
for ^^1. 

• We need a similar construction to Fukaya's extension [11] of Floer homology groups. 

The organization of this paper is as follows. In Section 121 we recall Fukaya's construction. 
We will basically follow [H] and [2], however, with some modification. In Section |3l making use 
of techniques developed in Section |2l we will introduce a variant li"'\Y; 7) of Floer homology 
groups for homology 3-spheres Y, loops 7 in F and a G {0, 1}. We define relative invariants 
^ I*^\Y;'y) for compact 4-manifolds Xq with boundary Y. We also prove a formula for 
similar to ([1]) (Theorem 13.261) . In particular, we deduce that when X is a connected sum 
Xq^Xi of closed, simply connected, non-spin 4-manifolds Xo,Xi whose Donaldson invariants 
associated with some S'0(3)-bundles are odd, the variant \E'^^ is non-trivial (Corollary I3.27p . 
In Section HJ we will prove the result on the non-existence of spin 4-manifolds with boundaries 
some homology 3-spheres and with some intersection forms (Theorem 14. ip . 

Acknowledgments . The author is grateful to Mikio Furuta for his suggestions. He would 
also like to thank Yukio Kametani and Nobuhiro Nakamura for useful conversations. 
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2. FUKAYA-FLOER HOMOLOGY 

Let X be a closed, oriented 4-manifold with 6+ > 1 and take a ?7(2)-bundle P over X 
and assume that the dimension of the moduh space of instantons on P is 2d for some integer 
(i > 0. We can associate cohomology classes of degree 2 on the moduli space to homology 

classes [S] G H2{X\ Z). Here S is a closed, oriented surface embedded in X which represent the 
homology class. The cohomology classes /^([S]) are the first Chern classes of the determinant 
line bundles of a family of twisted d operators on S. Roughly speaking Donaldson invariants 
are the evaluations of cup products yu([Si]) U ■ ■ ■ U on the moduli spaces. 

Suppose that we have a decomposition X = Xq Uy Xi and that the surfaces are split 
into two surfaces with boundary 7^ = along Y . We consider how to recover the Donaldson 
invariants of X from relative invariants of Xq and Xi. Fukaya's construction [TT] provide 
us suitable homology groups to define the relative invariants. In the construction, we need 
sections of determinant line bundles of families of twisted d operators on 7; x R which behave 
nicely on the ends of moduli spaces of instantons over y x R and satisfy certain transversality 
conditions. We call such sections admissible. The main point of this section is the construction 
of admissible sections of the determinant line bundles (Proposition 12. 8p . 

2.1. Outline of proof of gluing formula. In this subsection, we give a sketch of the con- 
struction of a gluing formula for Donaldson invariants. 

Let X be a closed, oriented, simply connected smooth 4-manifolds with fe"*" > 1 and odd, and 
P be a f/(2)-bundle over X with W2{P) non-trivial. We write Pdet for the ?7(l)-bundle over X 
induced by P, and fix a connection a^et on Pdet- We take a Riemannian metric on (7 on X and 
suppose that the virtual dimension of Mp is 2d with d > 0. Here Mp = Mp{g) is the moduli 
space of instantons on P which induce the connection a^et on Pdet- The moduli space Mp is a 
smooth manifold for generic metrics g, and a choice of an orientation of the space H^{X) of 
self-dual harmonic 2-forms on X orients Mp. The moduli space Mp gives us the Donaldson 
invariant 

This is defined as follows. 

Let [S] be a class in H2{X; Z). We denote by the space of gauge equivalence classes of 
framed, irreducible connections on the restriction P|s of P to S which are compatible with 
Odetls- We have the determinant line bundle 

£s = detlnd{a:^}[^]g^^ ^ i3s 

of the family of operators {c^A}[yi]eBE' Here 

is the d operator twisted by the rank-two complex vector bundle E associated to P and a 

square root of the canonical line bundle of S, and d\ is the adjoint. 

There is a natural action of SU{2) on B^, and the action of {±1} C SU{2) is trivial. Hence 
we have the S0{3) = SU{2)/ ± 1 action on B'^. The quotient space B^/SOi?)) is the space 
B'^ of irreducible connections on P|s which are compatible with Odetls- Since the action of 
{±1} C SU{2) on is trivial, we have the line bundle 

Cf := CflSO{?,) A . 
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Note that £f may not be a square of a genuine line bundle. Take a section of £f and 
denote the zero locus by Vg. We define Mp fl Vs by 

Mpnv^ = { [A] e Mp I G l^E }. 

Since the restriction of elements of Mp to S may not be irreducible, this is not well-defined. 
But we can avoid this problem by replacing E with a small neighborhood of S. (See [8].) 
Take d homology classes [Si], . . . , [S^] G H2{X\ Z). We can show that the intersection 

Mp n n • ■ ■ n v^, 

is transverse and finite for generic surfaces Si, . . . , and sections s^i, . . . , se^- Since Mp 
and l^E,; are oriented, we can associate each point of the intersection with a sign. We define 
\l/x,p([Si], . . . , [SJ) to be 1/2*^ times the number of points of the intersection counted with 
sign, i.e. 

vi/^,p([Si], . . . , [s,]) = ^#(Afp n i^E, n • ■ ■ n \/eJ. 

We can see that this is independent of the choices of metric and sections. 

Suppose that X has a decomposition X = Xq Uy Xi. Choose a Riemannian metric gy on 
Y and let go,gi be Riemannian metrics on Xo,Xi whose restrictions to Y are equal to gy- 
Assume that the dimension of the moduli space Mp is 2d with < d < 3. Take homology 
classes [Si], . . . , [S^^] of X represented by surfaces Si, . . . , S^^ which intersect with Y. Put 
S^ = S; n Xo, SJ' = S; n Xi and 7; = S^ fl F. We assume that 7; are diffeomorphic to 
for all /. We show how to compute \E'x([Si], . . . , [SJ) from data of Xo,Xi briefly under this 
situation. We use some facts about instantons which can be found in |7i|. 

Take a sequence {T""}^^ of positive real numbers which diverges to infinity. We have 
manifolds X"" = Xq U {Y x [— T", T"]) U Xi which are diffeomorphic to X. The Riemannian 
metrics gy, go, gi induce Riemannian metrics g"" on X°. Take instantons [A"] G Mp{g°'). 
Then there is a subsequence {[A° ]}«' such that 

KW([^o°°],---,m)- 

Here [A^] G M^^(/)(0)), [A^] G M^^{p{r - 1)) are instantons over Xq = ^0 U F x M>o, Xi = 
XiUF xM>o which converge to projectively fiat connections p(0), p{r — 1) at infinity. Mj^ (p(0)) 
is a moduli space of instantons over Xq with limit p(0) and similarly for Mj^^{p{r — 1)). For 
i = 1, ... ,r — 1, [Af] G My^]g(p(i — l),p(i)) = MyxR(p(i — l),p(i))/M are instantons over 
y X M with limits p{i — 1), p{i). ( The action of M on MyxR(p(i — l),p(i)) is defined by 
translations. ) Since dim Mp is less than 8, bubbling phenomena do not occur. As we will see 
below, we can take sections sf of — )■ Mp{g°') such that 

(2) sliiA'^]) s^Arn) M srMT]) ^ ■ ■ ■ ^ srMr-i]) ^ ^e-I^), 

where S^ = S^ U (7/ x R>o), S" = S" U (7; x R>o), = 7; x M, and sg/, s^,xir, s^, are sections 
of line bundles defined by families of twisted d operators on the surfaces. (See Definition 12.71 
and Proposition 12. 8[ ) Suppose that all [A"] lie in the intersection 
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then at least one of the components of the hmit of Ss([^"]) vanishes. A dimension counting 
argument shows that r is 1 and that 

m e M^^(p(0); L) = M^^(p(0)) n[]V^, [Ar] E M^^(p(0); L^) = M^^(p(0)) n f| V^,. 

Here L is a subset of {1, . . . ,d} and L'^ is its complement. We denote the number of elements 
of L by Then we also have 

(3) dimM^^(p(0)) = 2|L|, dimM^^(p(0)) = 2|L'^| 

and M^y(p(0); L), M^^(p(0); L'^) are finite (if the sections over the moduli spaces satisfy some 
transversality conditions and behave nicely on the end of the moduli spaces). A standard 
theory of gluing of instantons shows that 

Mpig'^)nV^,n---nV^, = [j\J M^^ (p; L) X M^^ (p; L^). 

L [p] 

Here [p] runs over the gauge equivalence classes of fiat connections satisfying This implies 
that 

(4) ^x,p(pi], . . . , [S,]) = J2Y1 #^Xo(P; L) ■ #M^^(p; L'^). 

L [p] 

From this formula, formal sums 

L [p] L [p] 

recover the Donaldson invariant. Here 

no{p;L) :=#M^^(p;L), n,ip; L^) := #M^^ (p; L^). 

We consider ipxo as an element of the vector space CFF(Y; 7) spanned by a set 

{ [p] ® 7l I and p satisfy }. 

The formal sums ipxo, '4'Xi depend on the metrics and sections. We will define a boundary 
map 

d : CFF{Y;j) CFF{Y;j) 

such that the composition d o d is identically zero, and show that dtpxo = and the class 
= bPxo] ^ HFF(Y;'y) = H{CFF(Y;'y),d) is independent of the metric and sections. 
There is a pairing 

< , >: CFF{Y- 7) ® CFF{Y; 7) — ^ Q 

such that [p] ® 7l and [p] ® 7l<= are dual to each other. We can see that the pairing induces a 
pairing 

HFF{Y; 7) ® HFF{Y- 7) ^ Q. 

The formula (j4]) implies 

^x,p([Si],...,[S,])=< ^Xo,^x, >. 

This is the gluing formula. 
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2.2. Fukaya's construction. Let F be a closed, oriented 3-manifold. Take a Riemannian 
metric gy on Y and a f/(2)-bundle Q over F, and fix a connection Odet on the ?7(l)-bundle 
Qdet induced by Q. We consider connections on Q which induce the fixed connection Odet on 
Qdet- Let Aq be the space of connections on Q with fixed determinant and Qq be the space of 
automorphisms of Q of determinant 1. The Chern-Simons functional is an S'^-valued functional 
on the quotient space Bq = Aq/Qq. The critical points are the gauge equivalence classes of 
projectively fiat connections. If the Hessian of the Chern-Simons functional at a projectively 
fiat connection is non-generate, we say that the projectively fiat connection is non-degenerate. 
For simplicity, we will refer projectively fiat connections over Y with fixed determinant as "fiat 
connections" . Throughout this section we assume the following hypothesis. 

Hypothesis 2.1. All fiat connections on Q are irreducible and non-degenerate. 

We are always able to perturb the Chern-Simons functional such that any critical points are 
non-degenerate. See [10], [7]. 

Let R(Y) = R(Y, Q) be the set of gauge equivalence classes of fiat connections on Q. It 
follows from Hypothesis 12.11 that R{Y) is a finite set. We define a Zg-grading function 5y on 
RiX) as follows. Let vr be the projection from y x M to F. Fix a fiat connection po on Q. 
For each fiat connection p on choose a connection Aq = Aq{p,pq) over F x M, which is 
compatible with 7r*adot, such that 

f 7r*p on r X (-00,-1), 
° \ n*po on r X (l,oo). 

Then we have an operator 

(5) Da, = d\ + ■ Ll{K\^^ ® tt^bq) L^((A^xr © Ky.w) ® ^*0q)- 

Here Qq is the bundle of trace free, skew adjoint, endomorphisms of the rank-two complex 
vector bundle E associated with Q. Under Hypothesis 12. 11 this operator is a Fredholm operator 
and we have the numerical index ind Daq € Z. We set 

5y([p]) = indD^o mod 8. 

We can show that this depends only on the gauge equivalence class of p (and po). 
For j ^'Lwe write CFj{Y) for the Q- vector space spanned by 

{\p\eR{Y)\5Y{\p])^3 mods}. 

Let d be an integer with 1 < < 3 and 7^ = 5^ be a loop in Y for / = 1, . . . , c?. We write 7 for 
{li}'i=v We define the Fukaya-Floer chain group CFF*(F;7) by 

d 

CFF,(F;7):=0 CF,_2^(r) ® Q < 7l >, 

/3=0 Lc{l,...,d} 
|L|=/3 

where 7^ := 7;^ ■ ■ ■ 7z^ G Sym Q < 71, . . . , 7^ > for L = {/i, . . . , //j}. We define a boundary 
operator 

d:CFF^{Y-^-^CFF^_,{Y-^ 
as follows. Take two generators 

[p] ®7L, G CF,_2/3,(r) ® Q < 7i, >C CFF,{Y-j), 
[a] ® 7L2 e CF,_2&-i ® Q < 7L2 >C CFF,-_i(r;7). 
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Then we have a moduh space MyxM(p, of instantons with hmits p, a and the dimension is 
2(/32 — + 1- We write My^^^ip, cr) for the quotient Myxr{p, where the action of M is 

defined by translations. When Li C L2, we define 

(6) < d{[p] ® 7lJ, M ® 7L. >:= " < ci(£f ) U ■ ■ ■ U Ci(£f/), [M° ,«(p, a)] > " , 

where {/i, . . . , = L2\Li and are line bundles defined by families of twisted B operators 
over 7;. x M. Since the moduli spaces are non-compact in general, we must specify the meaning 
of the pairing, and it will be done later. When Li is not included in L2, we define < d{[p] ® 
iLi), [o"] ®7l2 > to be zero. The matrix elements < d{[p] ®7li), [cr] ®7l2 > give the boundary 
operator d. We will show that d o d is identically zero. The Fukaya-Floer homology group 
HFF^.{Y] 7) is defined to be the homology group of {CFF^{Y; 7), 9). 

We give the precise definition of ([6]). To do this, we introduce some spaces of connections 
on y X M and 7/ x M. For fiat connections p, a on Q, take a smooth connection Aq = Ao{p, a) 
on 7r*Q as before. Let r > be a small positive number and we set 

^yxR(P> (t) := { Ao + a \ a e ll'^iA].^^ ® 'k*Qq) }. 

The wighted Sobolev space L4 ^ is defined as follows. Take a function Wj- onYxM. such that 

Wr{y, t)>0 for ^{y, t) eY xR, 
Wr{y,t) = e^\*\ for |t| > 1. 

For a smooth, compact supported section /, we define the weighted L|-norm by 

4 

\\f\\ll^ = T.\\<oiWrf)\\h. 
k=0 

The weighted Sobolev space is the completion of the space of smooth, compact supported 
sections. Note that since p, a are irreducible, every connection in Ay^^^p, a) is irreducible. 
By Hypothesis 12. we need not to introduce the weighted Sobolev space for the construction 
of the moduli space. However we need the weighted Sobolev space to define the determinant 
line bundles as explained below. 

We introduce a gauge group acting on the space of connections. We put 

Gyxr ■= { 9 ^ Ll i^^{AntTT*Q) \ dA^g ■ G }■ 

Then Qyxr acts on Ay^^^^p, cr) by the gauge transformations. We denote the quotient Ay^^/ Gyxr 
by i3yxig(p, cr) . For g G Qyxr, we have g{y,t) — )■ 1 or g{y,t) — )■ — 1 as t — )■ ±00. ( See Propo- 
sition 4.7 in [7]. ) We set 

Syxr ■= { g ^ Oyxr I lim g{y,t) = 1 }. 

We write By^^i^p, a) for Ay^^^p, o")/^yxR- Since Qyxr/Qyxr isomorphic to Z2 x Z2, we have 
a natural action of Z2 x Z2 on Bp^^{p, a), and By^^^p, a) is identified with By^^^p, cr) /Z2 x Z2. 

Let Myxr{p, cr) C By^^^p, a) be the moduli space of instantons with limits p, a. We can per- 
turb the instanton equation such that the operators Da are surjective for all [A] G My^^^p, cr) 
([lOil?]). For simplicity, we always assume the following. 

Hypothesis 2.2. The operators Da defined by are surjective for all [A] G My^^{p,cr). 
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Under this hypothesis, the moduh space My^]g(p, a) is smooth of expected dimension. 
Next we introduce spaces of connections and gauge groups over = 7; x M. Let Ai = Ai (p, a) 
be the restriction of the fixed connection Aq{p, a) to Ti. We set 

ArSP,cr) ■■={Ai + a\ae L^'"(A^^ ® 7r*0Q|rJ }, 

Si --={9^ 4^oc(Aut7r*Q|rJ | dA,g ■ g'' e hm g = l}. 

We denote the quotient space (p, cr)/^r, by i3r;(p, cr). Note that the restrictions of p, a to 
7i may be reducible, and hence some connections in Ari{p,cr) are reducible. 

We define the determinant hne bundle over -Br,(p, o"). We need a spin structure on F;. Since 
H^(Ti; Z2) is isomorphic to Z2, there are two spin structures on each F; (up to isomorphism). 
We fix a spin structure on 7; which represent the trivial class in the 1-dimensional spin bordism 
group. This spin structure induces a spin structure on F/. We use this spin structure. (We will 

explain the reason why we take this spin structure in Remark 12.31 below.) The spin structure 

1 

induces a square root K^^ of the canonical line bundle Kr^ . For connections A G (p, c") we 
have the twisted d operators 

Here L^'^''^''^^ are the weighted Sobolev spaces with weight function W!^ > such that 

W!^{y,t) = e^' for |t| > 1. 

We do not take the absolute value of t in the exponent this time. The operators are Fredholm 
operators for small r > 0. Since we have the universal bundles 

Er, := Ar,{p,a) Xgo^ (7r*E|rJ Br, x F^, 

we obtain complex line bundles 

Ci{p,a) = (detInd{(9A}[A])* — > Bri{p,a). 

Let f/ be the map from By^^^ip, cr) to ByXPi ^) defined by restricting connections to F^. Then 
we have a natural action of Z2 x Z2 on the pull-back riCf^ and the action of the diagonal 
Z2 = {±(1, 1)} is trivial. Hence we get the line bundle 

Cf\p,a) := r7£f (p,a)/Z2 x Z2 ^ S^xm(p,^) 

for each 

Remark 2.3. We explain the reason why we choose the spin structure on F; induced by 
the spin structure on 7^ representing the trivial class of 1-dimensional spin bordism group 
Qs^pm _ When we prove the gluing formula for Donaldson invariants, we consider F^ as a 
neck of a closed surface in a closed 4-manifold and we need the restriction of a spin structure 
on the closed surface, which is used to define Donaldson invariants, to F;. This spin structure 
is the spin structure induced by the spin structure on •ji which represent the trivial element in 

To define we need sections si = si{p, a) of Cf^i^p, a) which behave nicely on the ends of 
the moduli spaces and satisfy suitable transversality conditions. We briefly recall some basic 
definitions and facts which are relevant to the end of the moduli spaces. (See [7j for details.) 
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For a real number T let ct be the translation 

r X M — > F X M 
{y,t) ^ (l/,t + T). 

We call a sequence T of real numbers 

Ti < ■ ■ ■ < T,_i 

a translation vector. 

Definition 2.4. Let be a sequence in MyxR(p, cr)- We say that {[A"]}^ is weakly 

convergent to 

for some {[Ai], Zi) € [MyMpii ' 1). P(0) x Sym"'(F x M)) /M if there is a sequence {T°}„ of 
translation vectors with 

as a — > oo such that for each i the translates c^ti([A°]) converge to [A°°] over any compact 
sets of {Y X ]R)\Zj and \c^a{FAc)\'^dfiYxR weakly converge to I-Fa?" Po^/^rxR + ^^li ^zr Here 
dfiYxR is the volume form on y x M, = [zi, . . . , z^J and are the delta functions. 

Proposition 2.5. Any sequence in MY^^{p,cr) has a weakly convergent subsequence. 

Let {[^"]}a be a sequence of Myy^-^ip, cr) which weakly converges to (([Af^], Zi), . . . , {[A'^], Zr)). 
It follows from the additivity of the index of the operator Da that if the dimension of 
My^]g(p, cr) is less than 8 then Zi are empty for all i. In Section [2] and [3l we only con- 
sider the case when dim My^^^ip, a) < 8 and hence Zi are always empty. In Section HI we will 
Analise the end of moduli spaces under the situation where Zi are not empty. 

The end of the moduli spaces are described by gluing maps. We consider the case when 
are empty. Let Ui, . . . ,Ur be precompact, open sets of My^^ip, • • • , My^^{p{r — 1), o"). 
Then we have a gluing map 

Gl:U,x (To, oo) X ■ ■ ■ X (Tq, oo) x M^^^{p, a) 

for some To > 0. The map Gl is a diffeomorphism onto its image. 

Proposition 2.6. Let {[A°']}a be a sequence in My^j^lp, a) converging to some {[A'^], . . . , [A'^]) G 
Ui X ■ ■ ■ X Ur- Then for large a, [A°'] are in the image of the gluing map. 

We also have a gluing map 

Gl : Cf\p,pmu, ^ ■ • • ^ ^fipir - 1), cr)\u^ ^ Cf'{p, a)\,^Gi 
which covers Gl. For T = (Ti, . . . , T^-i) with Tj > Tq, we write GIt for the restriction of Gl 

to Ui X {Ti} X ■ ■ ■ X {Tr-l} X Ur. 

Using these definitions and facts, we state the properties of sections of the line bundles which 
are required to define (Ej). Let {[A"]}^ be a sequence in My^^^p, a) with limit • • • , [^J^])- 

By the above proposition, for large a, there are instantons [A"] G Ui and T" > Tq such that 

[A"] = G/([A?],Tf,...,T,%,K]). 
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Definition 2.7. Let si be sections of the line bundles Cf'^{p,a). Under the above situation, 
we say that converge to si{[A'^]) S ■ ■ ■ S s,([A~]) if 

- gIt^ (sKKi) K ■ ■ ■ K sKK])) II 

as a — 7- oo. Here || ■ || is the norm on £f ^(p, a) induced by the L^''^~^''^^-norms on the spaces of 

sections of K^^ (g) tt*E, Ap' (g) K^^ O tt*E. 

The following proposition is the key in this paper. 

Proposition 2.8. For flat connections p, cr on Q with dim My ^]g(p, a) < 8, we have sections 
Si{p,a) : My^jg(p, 0") — )■ Cf'^{p,a) which have the following properties: 

(a) For any sequence {[A°']}a in Myxjg(p, a) converging to some ([A?°], ■ ■ ■ , [A^]); 

si{[A-])^si{[AT])M...msi{[Ar]) 

in the sense of Definition \2. 7\ 

(h) Let Vi = Vi{p, cr) be the zero locus of s;(p, a). For L C {1, . . . ,d} with dim My^^^ip, cr) < 
2|L|, the intersection 

M^xm(p, L) := MO ,jj(p, a)nf]Vi 

is empty. 

(c) // dim My X (P; = 2|L|, i/ie intersection Myy^^iPyCr; L) is transverse and compact. Hence 
the intersection is a finite set. 

Here we introduce the following definition. 

Definition 2.9. If sections si{p, a) of Cf^i^p, a) have the properties in Proposition 12.81 we call 
them admissible. 

The proof of Proposition 12. 81 will be given in the following two subsections. In this subsection, 
we assume that we have admissible sections S;(p, cr) and define the boundary operator for the 
Fukaya-Floer homology groups. 

Let d be an integer with 1 < d < 3. For subsets Li C L2 C {1, . . . ,d} and fiat connections 
p, cr with dim My ^]g(p, cr) = 2\L2\Li\, the intersections My^]g(p, cr; L2\-Li) are finite by the 
property ^ in Proposition 12.81 Hence we can count the number of points in the intersections. 
They numbers give the definition of (jS])- More precisely we need to attach a sign ±1 to 
each point. However we mention nothing about signs. (The main purpose of this paper is 
to construct variants of Floer homology groups for 2-torsion instanton invariants. They are 
defined over Z2 and we do not need signs for the construction. ) 

Definition 2.10. Let 7 = {'~fi}f^i be a set of loops in Y, where d is an integer with 1 < d < 3. 
For integers /3i,/32 with < l3i < < d, take generators [p] G CFj_2i3iiX)-, M ^ C'-^j-2/32-i(^) 
and choose subsets Li, L2 of {1, ... , d} with \Li\ = f3i, IL2I = (32- Then we put 

#M0,^(p,a;L2\Li) if Li C L2 
otherwise. 



<a([p]®7ij,[cr]®7i, >: = 



We define d : CFFj{Y;j) CFFj_i{Y;j) by 

d{[p] 8) 7Li) := 5^ 5^ 5^ < ^([P] ® 7lJ, M ® 7L2 > M ® 7L2 

132 [a] L2 
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We prove the following. 

Lemma 2.11. d o d = 0. 

This is given by counting the number of the ends of 1-dimensional moduli spaces. Let f3i, P2 
be integers with Q < Pi < < d. Choose generators [p] G CFj_2i3iiX)i M ^ C-^j-2/32-2(^) 
and Li C L2 C with |Li| = IL2I = /32- We have a reduced moduh space 

My^jg(p, a) of dimension 2(/32 — + 1 (< 8). Suppose that we have a sequence {[A"]}^ in the 
intersection My-^^^p, cr; L2\Li) of formal dimension 1 which converges to some ([A^], . . . , [Af]) 
with r > 1. (Note that Proposition 12.81 does not assure that My ^^(p, a; L2\Li) is transverse, 
since dim My xjg(p, a) > 2\L2\Li\.) First we show that r = 2. Put 

L{i) = { / G L2\Li I = }. 

For I G L2\Li and all a, = 0, and converges to K • • ■ K ^^([A^]). 

Hence there is a number G {1, . . . , r} such that 

for each / G L2\Li. This means that / G L2\Li lies in L{i{l)). Therefore we have 

r 

/32-/3i = |L2\Li|<^|L(^)|. 

i=l 

Since are included in My^jg(p(i — 1), p(2); the intersection are not empty. The 

transversality condition (jb]) in Proposition 12.81 implies that 

2\L{{}\ + 1 < dimMyxM(p(2 - 1),pW) 

for each i. From the additivity of the index, we have 

2(/32 + 2 = dim My xr(p, a) 



^dimMyx]R(p(i - l),p(i)) 



i=l 



>j2mm\ + i] 



i=l 



>2(/32-/3i)+r. 

Therefore we get r < 2. We assumed that r > 1, so r = 2. 

Put L := L(l) IJLi. Using (jb]), in Proposition 12. 8^ we can easily see that 

m e M°,i,(p,p(l);L\Li), [AT] G M°,j,(p(l),a;L2\L) 

and that 

dimMO,i,(p,p(l)) =2|L\Li|, dim M° ,j,(p(l), a; ^L) = 2|L2\L|. 

Conversely for each [A] = ([^i], [A2]) G M^ ^^(P. P(l); A^i) x M^xuipi'^),^; L2\L), we 
have gluing maps 

Gl[A] : X (To, 00) X U[A2] — ^ M^^^{p,a), 



G/^ : £f (p,p(l))|^,,^, K£f2(p(l),a)|^,^, ^ Cf{p,a)\i^Gi 
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for some precompact open neighborhoods U[Ai], U[A2] of [Ai], [A2] and positive number Tq > 0. 
When U[Ai], U[A2] ai'e sufficiently small, the transversality conditions in Proposition 12.81 imply 
that intersections 

ieL\Li ieL2\L 

are transverse and that 

UiA,]n fl Vi = {[Ai]}, dU[A,]n fl ^ = 0' u,nVi = (li {leL^XL), 

ieL\Li ieL\Li 

f/[A,]n f Vi = {[A2]}, dU[A2]n f] Vi = (Is, U2nVi = % {ieL\L^). 

l£L2\L lGL2\L 

Put 



s[ := Gl{si{p,r)^si{T,a)) : U^a,] x (Tq, 00) x U1A2] ^ 'Cf\p,a)\i^ 



Gl 



14] 



and let V/ be the zero locus of s^. Then for each Ti > Tq we have 

iU[A,] X {T,} X U[A2]) n n V/ = {i[A,],T,,[A2])}, 

ieL2\Li 

9(f/[A,] X {Ti} X U[A2]) n n ^' = ^ 

and the first intersection is transverse. The condition (jaj) in Proposition 12.81 means that si{p, a) 
and s'l are close to each other on the end of the moduli space. Hence for Ti large enough, the 
intersection 

{U[A,] X {T,} X U[A2]) n n ^ 

l€L2\Li 

is transverse and consists of a single point which is close to ([Ai],Ti, [A2]). We consider a 
subset 

M' = M^,^ip,a■,L2\L^) \ (j [A^], Ti), 

([A1UA2]) 

of the moduli space. Here 

Ei[A,], [A2],Ti) = GllA,UA2]iU[A,] X (ri,00) X U[A2])- 

Then M' is compact and if we perturb sections si{p, a) outside of neighborhoods of the triples 
([74i],Ti, [A2]), M' becomes a smooth manifold of dimension 1. Moreover there is a natural 
identification 

(7) dM'= [j U U M^MP,r;L\L,)xM^,^iT,a;L2\L). 

/3 [t] L 

/3i</3</32 5y{[t])= ■^i|'=f5i^2 
j-2/3-1 mod 8 \^\-P 

By counting the number of dM' with signs, we get 

< 99(p(g)7Lj,a(g)7L, >= 

and this gives 

dod = 

as required. 

Definition 2.12. HFF,{Y;-f) := H,{CFF,{Y;-f),d). 
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2.3. Construction of s/(p, cr). In this section, we will prove Proposition 12.81 

First, for flat connections p, cr with dim My y.^{p,<j) < 8, we take locally finite open covers 

{?7A}AgA(p,o-) of MyxR(P)Cr) with U\ precompact as follows. We will use a partition of unity 

associated with the open cover to construct admissible sections. 

If dim My xjg(p, cr) = 0, Myx]g(p, a) is compact. Put Uq = Myx]jj(p, cr). Then {Uq} is the 

required open cover. Fix an integer m with 1 < m < 7 and suppose that we have open 

covers {?7a}a6A(p,o-) for flat connections p, a with dim My ^]g(p, a) < m — 1. We consider fiat 

connections p, a with dim My xjg(p, a) = m. For each A = (Ai,...,Ar.) G A(p, p(l)) x ••• x 

A(p(r — 1), a), we have a gluing map 

Glx ■■ Ux, X (Ta, oo) X • ■ ■ X (Ta, oo) x Ux^ M^^^{p, a) 

for some Ta > 0. We can suppose that Glx extends to the closures Uxi of Ux^■ For n = 
(ni, . . . , Ur-i) G (Z>o)''~S we put 

Ux,n := G/a (^Ux, X {Tx + ni,Tx + ni + ^) X ■■■ X {Tx + Ur-i, Tx + Ur-i + ^) x Ux,^ ■ 

This is a precompact, open set of Myx]g(p, a). We write N for the complement of the union 
of the sets Ux^n- 

iV:=M°,K(p,a)\ IJf^A,^- 

Lemma 2.13. Under the above notations and assumptions, N is compact. 

Proof. Let {[A"]}^ be a sequence in A^. By Proposition 12. 5[ there is a subsequence {[A"']}^' 
such that 

[A"']-^([An,...,m). 

Note that Zi are empty since the dimension of the reduced moduli space is less than 8. If 
r > 1, it follows from Proposition 12.61 that [A" ] lie in the image of Glx for large a' and some 
A. This is a contradiction to the fact that [A"''] are in the complement of the images of the 
gluing maps. 

□ 

Take a small neighborhood Uq of A^ in Myy^^ip, cr), and put 

m 

{Ux}xeAM ■■= {f/o} U U U U U {f^,!.}- 

r=l p{l),...,p{r~l) A n 

Then {Ux}xeAip,a) is a locally finite open cover of Myy^^ip, <j) with Ux precompact. We impose 
a condition on Ta, where A = (Ai, . . . , Ar). If Xi has the form Aj = (A'^^, . . . , X'^,;n[, . . . , n'^,_i), 
we suppose that Ta satisfies the inequality 

(8) Ta>^(Tv+<) 

k 

where A' = (A'^, . . . , A^,). This condition will be used for the proof of Proposition 12.81 

Next we construct admissible sections s;(p, cr) using a suitable partition of unity associated 
with the open cover {Ux}x- We again do it by induction on dim My ^^{p, a). Let p and a be 
fiat connections with dimMy^^{p,a) = 0. For generic sections si{p,a) on MY^Tg^{p,a), the 
zero locus are empty, and s;(p, a) have the properties in Proposition 12. 8[ Fix an integer m with 
1 < m < 7. Assume that for p, cr with dim My ^^^(p, cr) < m — 1 we have sections si{p,a) on 
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Myxjg(p, a) having the properties in Proposition 12.81 We need to construct admissible sections 
Si{p, a) for p, a with dim My ^]g(p, a) = m. We need some notations and lemmas. 

Take subsets / = {ii, . . . ,is}, J = {ji, ■ ■ ■ ,jt} of Ir-i := {1, ... ,r — 1} with I H J = (I), 
/ U J = Ir-i- Here ii < ■ ■ ■ < is, ji < ■ ■ ■ < jt- For n = (ni, . . . , n^-i) G (Z>o)^~^ we write 
Uj = {rii-^, . . . , njj, nj = {rij^, . . . , ra^J, and for A = (Ai, . . . , A.,.) we write |A| = r. 

Lemma 2.14. Let I, J be non-empty subsets of Ij—i as above. There is a partition of unity 
{fx}xeA{p,a) satisfying the following condition. For (A, n) G A(p, o") with |A| = r, there exists a 
positive integer N = N{X,nj) > depending on A and nj such that if nj > N for all j & J 
then fx n is identically zero. 

Using the partition of unity, we define sections si{p,a) on Myxjj(p, a) by 

m 

(9) Slip, ^) = E E E /a," Ghi^iip, P(l)) K ■ ■ ■ K sMr - 1), a)) . 

r=2 p(l),...,p(r-l) A,n. 

Here si{p, p(l)), . . . , si{p{r — 1), cr) are admissible sections. We will show that we can perturb 
s;(p, cr) on a compact set in My^]g(p, cr) such that si{p,a) have the properties in Proposition 

m 

To prove Lemma [2.14[ we need to remove extra open sets from the open cover {t/A}AgA(p,o-)- 
We consider the condition 

(10) t/A,nC U t/v,y. 

(A',n')GA{p,(7) 
lA'KIAI 

Put 

A' = k\p,a) := { (A,n) G A(p,ct) | Ux,n satisfies ([TO]) } 
A" = A"(p,a) :=A(p,a)\A'. 

By definition, {Ux}\£A" is still an open cover of Myxjg(p, a). Let {fx}xeA" be a partition of 
unity associated with {Ux}xeA"- For A G A(p, a), we define fx by 

/r if AG A" 



fx 



otherwise. 



Then {fx}xeA{p,a) is a partition of unity associated with {Ux}A(p,a)- We show that this partition 
of unity has the property of Lemma 12.141 It is sufficient to show the following: 

Lemma 2.15. Let I, J, (A, n) G A(p, a) be as in Lemma \2.14\ Then there is a positive integer 
N = N{X,nj) such that Ux,n satisfies / flOj) if Uj > N for all j G J. 

Proof. We give the proof in the case A = (Ai,A2,A3), / = {1}, J = {2}. (The proof in the 
general case is similar.) 

For [A] G Ux^n, we can write 

[A] = Glx{[A,],T,,[A2],T2,[A,]). 

Here [Ai] G Ux, , Tx + ni<Ti<Tx + ni + ^. As T2 ^ oo, [A] converges to ([^12], [A3]) 
for some [^12]- Hence there is a positive real number T2 = T2°([y4i], [A2], [y43],Ti) such that 
if T2 > T2 then [A] lies in the image of a gluing map. Therefore [A] is included in Ux\n' 
for some (A',n') G A(p,a) with |A'| = 2 (< |A|). Since Ux„Ux^,Ux3, {Tx + ni,Tx + rii + |) 
are precompact, we can take T2 uniformly with respect to ([^i], [^2], [^sl^^i)- Therefore we 
obtain the statement. □ 
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Define sections si{p,a) by Then we have: 

Lemma 2.16. The sections si{p,a) satisfy in Proposition \2.8\ . 

The proof will be given in Subsection 12.41 Here we prove that we can perturb the sections 
si{p,a) such that the sections become admissible, assuming Lemma f2.1& We show the next 
lemma to do this. 

Lemma 2.17. For each L G {I, . . . ,d} with dim Myy^^ip, a) < 2\L\, the intersection My^]g(p, a 
is compact. 

Proof. If My^]jj(p, cr; L) is not compact, there is a sequence {[A"]}^ in Myy^^ip, cr; L) converging 
to some ([Af^], . . . , [A'^]) with r > 1. From Lemma \2.16\ we have 

si{[A''])^si{[A^])M---msi{[A'^]) 

as a — oo. Since Si([y4"]) = for / G L, we have 

si{[Ar])^---^sii[Ar])=0. 

This means that for each / G L, there is some i{l) such that S;([y4^j-|]) = 0. Put 

m ■.= {ieL I si{[Ar]) = }, 

then we obtain 

r 

L = \jm. 

i=l 

Hence we have 



(11) \L\<j2\m\- 



i=l 



Using this inequality, we show 

(12) dimM°^^(p(zo - l),p(^o)) < 2\L{to)\ 

for some io- If not, we have 

dimMyxR(p(2 - 1),pW) > 2\L{{)\ + 1 

for all i. From ffTTl). 



dimMyxM(p,a) = ^dimMyxR(p(« - 1),pW) > J2^2\L{t)\ + 1) > 2\L\ + r > 2\L\ + 1. 



i=l i=l 



This is a contradiction since we assumed that dim My^]g(p, cr) < 2\L\. We have obtained (fT2l) . 

By the hypothesis of induction, si{p{io — l),p(zo)) satisfy (jbj) in Proposition 12.81 Hence the 
inequahty (IT^ means that 

On the other hand, [A'^] lies in My^^ipiio — 1), p(^o); -^(^o)) by the definition of L{io). We 
have obtained a contradiction. Therefore My^]g(p, a; L) is compact. 

□ 
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Proof of Proposition \2.8\ . 

When dim My^]g(p, cr) = 0, generic sections s/(p, a) are admissible. Let m be an integer with 
1 < m < 7 and suppose that we have admissible sections si{p, a) when dim My-^^^ip, cr) < m—1. 
Let p,a be flat connections with dim My ^]g(p, o") = m. It follows from Lemma 12.161 that 
the sections si{p,a) defined by ([9]) satisfy (Ej) in Proposition 12.81 From Lemma [2.171 the 
intersections My^]g(p, a; L) are compact if dim My ^jg(p, a) < 2\L\. Hence perturbing Si(p, cr) 
on a compact set of My^]g(p, a), the transversality conditions (jb]), (icj) in Proposition 12.81 are 
satisfied. Since the region where s;(p, a) are perturbed is compact, the perturbed sections also 
satisfy Therefore the perturbed sections are admissible. 

2.4. Proof of Lemma 12.161 It remains to show Lemma 12.161 For simplicity of notations, 
we give a proof of the case when [A"] converges to ([^f^], [^2°])- ^o do this, we show three 
lemmas. 

Let converge to {[A^], [A^]) G M^^^{p,p{l)) x ^^{p{l) , a) . In the first lemma, we 
consider the situation where the connections [A'^] can be written as gluing of three instantons. 
For example, such a situation occurs if [A°'] are given by 

[A^] = Gh{[A,],T„[A2],T,^,[A,]). 

Here A, [Ai], [A2], [A3] and Ti are independent of a, and 00. 
Assume that [^4"] G Glx^ with |A"| = 3. We can write 

for some ] G Uxc {i = 1, 2, 3) and S°' > T\a [j = 1,2). The first lemma is the following. 

Lemma 2.18. There is a subsequence {[A"']}a' which satisfies the following. 

(1) }^ are independent of a' . We denote A" by X = (Ai, A2, A3). 

(2) For all i, [Bf] G U\. converges to some [-Bf°] G [/a, • 

(3) We have either Sf < 00, S^' ^ 00 or 5f' ^ 00, ^2"' ^ <oo. 

We can show similar statements to this lemma when we assume |A"| = £ with £ > 4. As we 
will see later, these statements imply that when [A"] splits into two instantons there are no 
terms with | A| > 3 in ([9]) for large a . 

In the second lemma, we consider the difference of two gluing maps. Let {[A"]}^ be a 
sequence in M^^^Xp^^^) converging to some ([A^], [A^]) G M{^xig(p, p(l)) x M^^jgfpjl), o"). 
Let f/j, U[ be precompact open neig hborhoods of in M^^^{p{i - l),p(i)) for z = 1,2. 
(Here p(0) = p, pi2) = a.) Then we have two gluing maps 

Gl:U,x (To, 00) X t/2 M° ^jj(p, a), 

Gl' : U[ X (To, 00) xU!,-^ M^^^ip, a). 

When a is sufficiently large, we can write 

[A-] = G/([A?],T°, [A^]) = G/'([A'°],T'", [A',-]) 

for some [A^] G Ui, G T",T'" > To. 

Lemma 2.19. Under the above notations, we have 



lim 

a— >oo 



GIt^ (sK[A?]) K si{[A^])) - {si{[A[^]) K s^iA',-])) 



0. 



FLOER HOMOLOGY FOR 2-TORSION INSTANTON INVARIANTS 



17 



As before, let {[A^jjo, a sequence of instantons which converges to a pair ([v4[^], [A'^]) e 
M^^^{p,p{l))xM^^^{p{l),a). For A with [A"] G Im G/a and with |A| = 2, we have [A°(A)] G 
Ux,, T^iX) > Ta such that 

K] = G/a([A?(A)],T"(A),[A-(A)]). 
In the third lemma, we consider the behavior of [A" (A)]. 
Lemma 2.20. 

(1) There is an integer ao > such that if a > ao, then [A"(A)] have the same flat limits as 
[A^]. That IS, [4"(A)] G M^^^{p{i - l),p{i)) for a > ao- Here p(0) = p, p{2) = a. 

(2) For any positive number 6 > 0, there is some as > ao independent of X such that if a > as 
we have 

d{[At{X)],[Ar])<S 

for all X with \X\ = 2, [A"] G ImG^A and i = 1,2. Here d{-, ■) is the metric on Myxjg(p(2 — 
l),p{i)) induced by the L^^^ -norm. 

Before we prove these lemmas, we show Lemma 12.161 using the lemmas. 

Proof of Lemma \2. 1 61 assuming Lemma \2.1t^ \2.iy\ and \2.2(h 

Let {A"]}q, be a sequence in My^j^ip, <j) with limit ([Af ], [^2°])- Let Ui be precompact open 
neighborhoods of [A°°] for i = 1,2. Then we have a gluing map Gl from Ui x (To, oo) x U2 to 
Myxjj(p, a). For large a, we have 

[A"] = G/(K],r",K]) 

for some [Af] E Ui, T" > Tq. We need to show 

sKK]) - GIt^ {si{[A^]) K si{[A^])) II = 0. 
First we prove the following claim. 

Claim 2.21. Let {/a}a be the partition of unity as in Lemma [2.141 Then there is a positive 
integer ai > such that if a > ai, then /A,n([A"]) = for all {X,n) ^ A(p, a) with |A| > 3. 

Proof. Otherwise we have a subsequence {[A"']}^' such that f^a' ^a'{[A°'']) ^ for some 

(A" ,n°') G A(p, cr) with |A° | > 3. From the additivity of index, we have 

|A°'| < dimMyxK(p,a). 

Hence we may assume that |A" | are independent of a' . For simplicity, we suppose |A" | = 3. 

By Lemma 12.181 we can suppose that A° are independent of a' and we denote it by A = 
(Ai, A2, A3). Since the supports of /a„c«' are included in ?7;^„q' and /a„q'([^"']) 7^ 0, \A^'\ lie in 
U^^^^a'. Hence there are [Bf] G Ux. for z = 1, 2, 3, and Sf > Ta for j = 1, 2 such that 

[A^'] = GhilBf], Sf, [B^'], S^', [S3"']). 

We can assume that [Bf] converge to [B°°] for i = 1, 2, 3 and that T"' — T^ < 00, Tf — )■ 00 
by Lemma I2.18[ The n°' are pairs (n" , ) with 

T, + nf <Sf <T, + nf + 

Hence nf — )■ < 00, nf — )■ 00. Put / = {1}, J = {2}, then we have 

nf > N{X,nf) 



lim 

a— ^00 
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for large a'. Here N{X,nf) is the integer which appeared in Lemma [2.14[ Hence fxn^' are 
identically zero for large a'. This is a contradiction. □ 

It follows from this claim and (|9]) that 

(13) Si{[A"]) = J2 /A,n"(A) ^A,T^(A) {^li[Am]) ^ ^KK(A)])) 

A=(Ai,A2) 
[A"]&mGlx 

for a > ai. For i = 1,2 and 5 > 0, we put 

U,,s := { [A] G M°,^(p(^ - l),p(z)) I [4-]) < S }■ 

The open covers {[/A}AgA(p(i-i),p{j)) are locally finite, hence if 6 is sufficiently small then the 
numbers of open sets U\ with U\ fl Ui^s non-empty are finite for all i. We fix such 5 > 0. 
By Lemma [2.201 there is a positive integer as such that if a > as, we have [A°'(A)] G Ui^s 
for all A. Let {f^Ai(p)}p=i be the sets of open sets which intersect Ui^s for each i, and put 
a'g := max{ai, as}. Then A in (fT3|) take the form of 

A(pi,P2) = (Ai(]5i), A2(p2)) (1 < Pi < gi, 1 < P2 < ^2) 

for a > a'g. 

By Lemma [2.19[ for any e > 0, there exists a{pi,p2; e) such that if a > a{pi,p2; e), we have 



Here [A"(l'i,P2)] = [^r(A(Pi,P2))], ^"(^1,^2) = T"(A(pi,p2)). We put 

a{e) := max{ a^, a(pi,p2; ^) I 1 < Pi < gi, 1 < P2 < ^2 }• 

Then for a > a{e) 

\\si{[A-])-Gh^{s,{[A'^])^s,{[A^]))\\ < Yl /A,n^(A)(K])5 = e. 



< e. 



A=(Ai,A2) 
[A^lelmGiA 



Therefore Si([y4"]) converges to si{[Af]) KI 



□ 



Here we prove Lemma 12.181 and I2.2U[ The proof of Lemma 12.191 will be given in the next 
subsection. 

Proof of Lemma \2.1S\ 

Let [A"'] , [5"] and S" be as in Lemma 12.181 There is a subsequence { [A" ] such that 
[Bf] converge to ([-Bf^], . . . , for all i. We will show that = 1 for all i. This implies 

(1) and (2) since the oven covers {Ux}x are locally finite. 

If not, Tig > 1 for some ig- For simplicity, we assume ri = 2, r2 = = 1. Since [B2 ], [B^ ] 
converge and {Ux}x are locally finite, we may suppose that A2 , A3 are constant. We write 
A2, A3 for Af', Af'. On the other hand, the sequence {[B'^']}a' has the limit {[B'^], [B^])- We 
may suppose that A" take the form of A" = (A'^, A2; n°' ) with — )■ 00. Put A' := (A'^, A2). 
We assumed that Tx satisfy the inequality ([H]) for all A. Hence we have 

Ty + n"' < Tx^ < Sf 

for j = 1, 2. In particular, S"' — 00 for j = 1, 2. This means that [A"] converge to some 
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This is a contradiction since [A"] — )■ ([^4^], [A^]). Therefore we obtain ri = r2 = = 1. 

Lastly we prove (3). If (3) does not hold, then we may suppose that 5" — S°° < oo 
for j = 1,2 or S"' — )■ oo for j = 1,2. In the first case, [A"''] does not split into instantons 
as a' — )■ oo. In the second case, [A°''] splits into three instantons as a' — )■ oo. Both cases 
contradict the fact that [A°'] split into two instantons. 

□ 

Proof of Lemma \2.2(A 

Let [A"], [v4"] and T"(A) be as above. Suppose that (1) does not hold. Then we have a 
subsequence {[A'^'jja' and A"' G A(p,p"'(l)) x A(p"'(1),(t) with [A"] G \mGl^^> and p"'(l) ^ 
p(l). By Hypothesis 12. H the set of gauge equivalence classes of fiat connections is finite. Hence 
we can suppose p" (1) are independent of a' . We write p'(l) for p" (1). As in the proof of 
Lemma |2.18[ we can see that there is a subsequence (still denoted by [A"']) such that [Af'(A" )] 
converge to some [^^°°], A" are independent of a' and that T"'(A" ) diverges to oo . This means 
that 

W] ([A'n, [A;-]) G M°,j,(p,p'(1)) X M°,j,(p'(l),a) p'(l) ^ p(l). 

This is a contradiction to the fact that [A"] converge to ([Af ], [A^]) G My-^^^p, p{l)) x 
My^]g(p(l), 0") and we have shown that (1) holds. 

If (2) does not hold, there is a subsequence {[A"']}^/ and A° G A(p, p(l)) x A(p(l),cr) with 

(14) rf([<(A-')],m)>5. 

As before, we can deduce that [Af (A" )] converge to some [A^] for i = 1, 2, A° are independent 
of a', and that T"'(A" ) diverges to oo. Hence [A"-'] converge to ([^4'^^^], [^2°°]). From f|T^ . we 
have rf([4~], > 5. In particular, [Af] ^ This is a contradiction. 

2.5. Evaluation of difference of two gluing maps. In this subsection, we prove Lemma 
12.191 We need to compare two gluing maps defined on different regions intersecting each 
other. To do this, we must explicitly see the construction of gluing maps. We give outline of 
the construction following ^ before we prove Lemma [2. 191 

First note that elements in Myy^^i^p^cr) can be considered as elements in MyxK(p, cr) with 
center of mass 0. Here the center of mass of [A\ G MyxR(p, cr) is defined to be 

/ ^I-^aI^ dpYxR- 
JyxR 

Let Xi,X2 be two copies of y x M and for T > put Xi{T) = Y x (-oo,T), X2{Y) = 
Y X (— T, oo). For each T, we have an identification 

(p: r X (T, 2T) — > Y X (-2T, -T) 
iy,t) ^ iy,t-3T). 

Gluing A:i(2r) and A:2(2T) through Lf, we obtain a manifold = Xi(2T) U<^X2(2T). We 

have a natural identification between X*^^^ and F x M such that (y, 3T/2) G Xi(2T) C 
corresponds to {y, 0) G F x M. 

Let p = p(0), p(l), 0" = p(2) be flat connections and choose [Ai] G M^^(p, p(l)), [A2] G 
(p(l); cr). Here Ai, A2 are instantons on Xi, X2 with center of mass 0. We write p(0), p(2) 
for p, a respectively. Then we can write 

Ai = Bi -\- ai, 
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where are connections with 

Bi -- 



p{i — 1) on F X (—00, —1) 
p{i) on y X (1, 00) , 



and ttj are 7r*0Q-valued 1-forms on Xi with 

\V^a,{y,t)\<Cke-'' (^A; G Z>o) 
for some Cfc > 0, 5 > 0. Fix a smooth cut-off function % : F x M — )• [0, 1] such that 



We define A'^ by 



1 on F X (—00, 0), 
on Fx (1,00). 



a; = 5i + x{t - T)ai, = B2 + xi-t - T)a2. 
These connections give a connection A' on X*^-^) = y x R such that 

r c\^^/^{A[) on r X (-00, -f + f] 

(15) A' = I p(l) onFx (-f + -f) 

[ 4^/2(^2) on F X [f - 1,00) . 

Let be the self-dual part of the curvature of the connection on n*QQ induced by A'. Then 
we have 

\\Fa'\\lI{YxR) < const e"^^. 
That is, A' is an almost instanton. To obtain a genuine instanton, we consider the equation 

for a small 7r*0Q-valued 1-form a. The equation can be rewritten as 

(16) c/+ a + (a A a)+ = 

By Hypothesis \2.2\ Da^ = d*^. + d\_ have right inverses Pi. When T is large enough, we can 
construct a right inverse P = Pt of D^' by using Pi,P2 and some cut-off functions. We can 
see the operator norm ||P|| of P is bounded by 2(||Pi|| + ||-P2||)- If we substitute a = Pcj) into 
(fT6!) . we get the equation 

4>+{p^A p^y = -F+ 

for (j) G L|(Ayx]fj ® vr*0Q). We can see that for large T > this equation has a unique solution 
(f) = (pT with — > as T — 7- 00. Hence for large T we get an instanton of the form 

(17) = A' + a 
with 

(18) lim a = 0. 

T— >-oo 

For Ui C Mx.{p{i — l),p(z)) precompact open sets, the construction can be applied to all 
{[Ai], [A2]) E Ui X U2 and we obtain the gluing map 

Gl:UiX (To, 00) X f/2 ^ M° ^^(p, a) 

for large Tq. (We can take Tq uniformly since Ui, U2 are precompact.) 
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More precisely, we must slightly translate A*^^^ to make the center of mass be 0. Let rriT 
be the center of mass of A*^^\ Then the translate c*_^^{A*^^^) is an instanton with center of 
mass 0. The precise definition of the gluing map is 

Gl{[A,],T, [A,]) = [c*_^^{A*^^^)]. 

We can easily show 

lim rriT = 

T->-oo 

and it does not matter even if we assume that itlt = in the proof of Lemma I2.19[ Therefore 
we will drop c!L„^ from notations in the proof. 

Let 7 = S*^ be a loop in Y and put F = 7 x M. We see outline of the construction of the 
gluing map 

Gl : Cf{p,p{l))\u,^Cf{p{l),cr)\u, ^ Cf{p,a)\,^Gi 

covering Gl. Let Ai,A2 be instantons on Xi,X2. For simplicity of notations, we suppose that 
are surjective for i = 1, 2. Let Fi, F2 be copies of F and fix cut-off functions 7^ = on Fj 
with 

IM7.IIC0 = O (^i^ , supp(7.) C F,(2T), (71)^ + (72)' = 1 on F#(^). 

Here supp(7j) is the support of 7^, and Fj(2T), F**^-^) are defined as before. In the third 
equation, we consider 7^ as functions on F**^-^) in the natural way. Take fi G ker (9^. for i = 1,2 
and put 

(19) /' = C_3T/2(7l/l) + C3T/2(72/2)- 

Then we can show that 

\\dA*mf'\\^u-..r, < £(T)(||/i||^.,(-.,.) + ||/2||^2,(-.,.)), 
e{T) — ^ as T ^ 00. 

We may construct a right inverse Qt from right inverses Qi of d^. for large T. The difference 

(20) f*^^^ = f'-QTdA*mf 

lies in ker(9^#(T). The operator norm \\Qt\\ of Qt is bounded by 2(||Qi|| + IIQ2II), hence we 
have 

(21) QTdA*mf'^0 

as T — 00. It can be shown that for large T the map (/i, /2) ^ /#(^) is an isomorphism from 
ker 9^j©ker to ker Bj^if^m and induces the isomorphism from CyXPi P(l))[Ai]®'^r( (p(1)5 <^)[A2] 
to £r,(P; cr)[yi#(T)]- Applying this construction to all connections in Ui x f/2, we get the map 

Gl. 

When ^"^^ compact, we can choose maps 

Ui : M"' — y n'^^f{E^Kl) 

such that dAi ® surjective. Applying the above method to these operators, we obtain 

the gluing maps. 

Proof of Lemma \2.iy\ 
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Assume that we have a sequence {[A"]}^ in My^^^p, a) which converges to some ([^^], [^2°]) ^ 
Myx]g(p, p(l)) X My^]g(p(l), (j). Take precompact, open neighborhoods Ui,U- of [A°°]. Then 
we have two gluing maps 

Gl:UiX (To, 00) X t/2 M° ,^(p, a) 

Gl' : U[ X (To, 00) X f/^ M° ,^(p, 

For large a, we have two different expressions of [A°'\. 

[A^] = G/(K],T°, [A^]) = Gl'{[A[%T'-, [A',^]). 

We can take representations Af, of [Af], [A-"], [A°°] such that A ° converge to 

A°°. For simplicity, we assume that are surjective for all [Ai] G f/j. Then what we must 
show is that 

lim WGlMfi, f2) - G'/;,.(/;°, = 

a— >oo 

for sequences {/f }q, {fi'^}a of kerc^A^, kerS.'c which converge to some /°° G 'keTdA°° in the 
Lg'^'^'^^-norms. From (HI), ([20]) and (EI]), we have 



^^^^ G/;,.(/;°, f^^) = c ,Mlif'n + C3^(72/;") + ^7'" 

2 2 

for some sections (7°, with g°'.,g'^ 0. Here we show that the difference T" — T'" goes to 
zero as a — 00. 

Lemma 2.22. lim^^^oo - T'°| = 0. 

Proof. Since [A"] converges to ([A^^], [^2°])) have real numbers S'f , S2 with — 5'f — > 00 
such that for any compact sets in F x M, 

(23) cs^i\A'^\K]) [^r 



r 2,T 
m L4' . 



On the other hand, [A"] can be written as Gl{[Af\,T°', [Af]). It follows from ([15]), ([I?]) and 
TED that 



(24) c-^{[A''\k\)^[AT\k]- 
Comparing f[23l) with f[24l) . we have 

lim |5i"-(3T"/2)| = 0. 

a— >oo 

Similarly we have 

lim IS-f - (3T'"/2)| = 0. 

o— >oo 

Hence we obtain the required result. 
From ( l22l) . we get 

< iic_3T^,(7i/r) -c_3T^,(7i/;")ii + iic_3T^(7i/;") -c_3^(7i/;°)ii+ 
iic3T^,(72/2°) - c3T^(72/;")ii + iic3T^,(72/;") - C3,.„ (72/;")ii + + \\g'% 



□ 
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Here || ■ || is the L^'*^ '^'^^ norm. The first term on the right hand side is equal to 

ii7i(/r-/;")ii. 

This is bounded by || /f — || , and || /f — || converges to zero since both of /f , converge 
to f^. Hence the first term goes to zero. 

The second term is equal to 

ii7i/;"-c,.(7i/;")ii, 

where 5° = 3(T" - T °)/2. It follows from Lemma [222 that ^" goes to zero. Since is 
finite and is bounded, for any e > there exist T(e) > independent of a such that 

||/r|yx(-oo, -r(e))|| < g, ||/nyx(T(e), oo)|| < g 
||c5Q(/i°°)|yx{-oo, -T(e))|| < g, ||Q-(/r')|yx(T(e), oo)|| < g- 

Since converge to f^, we have a{e) > such that if a > a{e) then 

ii/;"-/rii <|- 

Thus we have 



Yx{-oo, -T{e)) 

< ||7l/l°|yx(-oo, -T(e))|| + ||c<5"(7l/l")|rx(-oo, -T(e))\\ 

< ~ + ll/nyx(-oo, -T(£))|| + ||c5<^(/l" - fl 



|c<5"(/D|yx(- 



oo, -T{e))\ 



< 



Similarly 



Yx{T(e), oo) 



< 



Since 7i/{°° is uniformly continuous on y x [—T{e),T{€)] and converge to zero, we have 



(7i/r-c^.(7i/r)) 



yx[-T(e),T(e) 



< 2T{e)M{e) max - cs^lifni ^ 

Yx[-T{e),T(£)] 

as a — 7- oo. Here M{e) is the maximum of the weight function W!^ over Y x [—T{e),T{e)]. 
Hence we get 



< 



(71/;^ - 7i/r) 





Yx[~T{e),T{e)] 
yx[-T(e),T(e); 



(7i/r-c5.(7i/r)) 



Yx[-T(e),T(e)] 
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as a — 7- oo. Therefore 
limsup ||7i/{" - 

< limsup 



-c..(7i/;")ii 
(7i/;"-c5.(7i/;")) 



(7i/^-Q.(7i/;")) 



Yx{~T{e), T{s)) 



+ 

yx(-oo, -T(e)) 



Yx{T{e), oo) 



< e. 



Since e is arbitrary, the norm of difference ||7i/{° — csai'Jifi")]] goes to zero. Thus the hmit 
of the second term is zero. Similarly the limits of the third and fourth terms are zero. Since 
g°', (yf " — )• 0, we have 



hm \\GlT4fi,f2)-GlMfi'',f2 







as required. 



2.6. Well-definedness. A priori, Fukaya-Floer homology groups seem to depends on the 
choices of Riemannian metric on Y and sections of Cf'^{p, a). But we will prove that Fukaya- 
Floer homology does not depend on these choices up to canonical isomorphism. In the proof 
of well-definedness of usual Floer homology groups, we need the functorial property of Floer 
homology groups with respect to Riemannian bordisms. We shall generalize the functorial 
property to Fukaya-Floer homology groups and prove the well-definedness by using this prop- 
erty. 

The main statement of this subsection is 

Proposition 2.23. Take two Riemannian metrics go,gi on Y and two sets {si{p,a)}p^fjj, 
{s'i{p, (j)}p^cT,i of admissible sections. We write HFF^{Y] 7), HFFl{Y; 7) for Fukaya-Floer ho- 
mology groups associated with the metrics and sections. Then we have a canonical isomorphism 
between HFF^iY]^) and HFFl{Y]-i). 

Assume that we have the following data: 

• two oriented, closed Riemannian 3-manifolds (Fo^S'o)) (^iiS'i)) 



• t/(2)-bundles QcQi over Yq.Yi satisfying Hypothesis 12.11 

• sets of loops 7 = {7z}f=i, V = {7i'}f=i, 

• a Riemannian bordism (X, G) between (lo^S'o), (Xi^gi)^ 

• a [/(2)-bundle P over X with P\yq = Qo, P\yi = Qi, 

• oriented, compact surfaces S/ embedded in X with boundary 7^ ]J 7^. 

We introduce the following notations: 

• X := X U (Fo X M>o) U (Fi x M>o), 

• the extension P of P to X, 

• the extension G of G to X, 
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• ±1 := Si U (7i X R>o) U (7,' x R>o). 

Let p, a be flat connections on Qq, Qi respectively. We denote the moduli space of instantons 
on P with limits p,a by Mj^{p,a). Using families of twisted d operators on S;, we get the 
line bundles C^^{p,a) over M^(p, a). As the proof of Proposition I2.8[ we can show that for 

p,a with dimM;5^(p, 0") < 8 there are sections si{p,a) of C'?^{p,a) satisfying the following 
conditions: 

• Let { [A""] }a be a sequence in (p, a) converging to 

{[An...,[Ar],[A^],[A'n,...,[K^]), 

where 

meM°„,Jp(z-l),p(^)), 

[A-]GM^(p(r),p'(0)), 
[A'r]^M^^,^{p'{^-l),p'{^)). 

Then we have 

siilA^]) sii[A^]) m si{[A'^]) M s\{[A'n) s[{[A',r]). 

• For L C {I, . . . ,d} with dimM^(p, a) < 2\L\, the intersection Mj^{p, a; L) = Mj^{p, a)r\ 
Hi^lVi is empty. 

• For L with dimM^(p, cr) = 2|L|, the intersection Mj^{p,a; L) is transverse and com- 
pact. Hence the intersection is finite. 

Lemma 2.24. Using the data (X, G, S;, s;), we can define a homomorphism 

with the following property: 

• The homomorphism Q is a chain map. That is, dio = Q o Oq. 

• The induced homomorphism C* : HFF^{Yq]'~^) — )■ HFF^(Yi]'y') is independent of the 
metric G and the sections si{p,a). 

• Assume that we have a bordism {X', G', P', {^'i}f=i) from (Yi, gi, Qi, 7') to other 4-tuple 
{Y2,g2,Q2,Y)- Let C be the map from GFF{Yi;i) to GFF{Y2;Y). For T > we 
define a bordism X**^^^ between Xq, X2 to be 

X(^) := XU (Fi X [0,r]) UX'. 

The metrics G, G' naturally induce a metric G*-"^^ on X*^"^) and we have surfaces = 
U 7' X [0, T] U with boundary 7^ ]J 7". So we have the map C(^) from GFF,{Yo]-f) 
to CFF^{Y2\i') . Then for large T > 0, 

We can derive Prop osition l2.23l from Lemma [2.241 Take two Riemannian metric go,gi on Y 
and two sets {si{p, cr)}i, {s[{p, a)}i of sections of Cf^lp, a). Put X = F x [0, 1], = 7; x [0, 1], 
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P = 7T*Q. Here vr is the projection from X = y x [0, 1] to Y. Choose a Riemannian metric G 
on X with G\yo = do, G\yi = gi- These induce the maps 

C.: HFF,{Y-j) HFFliY-^, 

C: HFF:{Y;j) HFF,iY;j). 

On the other hand, the metric Gq = go + dt^ and the pull-backs p*{si{p,a)) of si{p,a) by 
p : Myxr{p, cr) — ^ M^YxR^Py ^) induce an endomorphism ('J of HFF^.{Y; 7). It follows from the 
construction of C, we will see below that is the identity map. The third part of Lemma [2.241 
implies 

CoC. = C = ^d. 

Similarly the composition C* °C is the identity map of RFF'^iY] 7). Thus C* is an isomorphism 
from HFF^Y; 7) to HFFl{Y; 7). 

We begin the proof of Lemma [2.241 Recall that the degree (5yo([p]); '^i^i(M) defined by 

^YoUpI) = in(iDA(p,po) mod 8, 
= iridDA(cT,pi) mod 8 

for some fixed flat connections pcPi over Y(),Yi. For simplicity of notations, we suppose that 

mdDA(po,pi)=0 mod 8. 

Here A{p, a) is a connection on X with limits po, pi. Then the dimension of Mj^{p, a) is equal 
to 5yi(cr) — Syo{p) modulo 8. 

For < /So < /3i ^ d, Lq C Li C {!,..., d} with |Lo| = (3o, \Li\ = [3i and generators 
[p] e CFj_2/3o(yo)) M £ CFj_2i3iiXi)^ the intersection Mj^{p,a] Li\Lq) is finite. We define 
<C([p]®7Li), M®7L2>by 

< C([p] ® 7lo)> M ® 7li >= #^x(P' ^A^o). 

Then the matrix elements give the map ( : GFF^,{Yq; 7) — )■ CFF^.(Yi; 7'). That is, 
cap] ® 7Lo) = 5Z 5^ 5^ < C([P] ® 7lo), M ® 7i, > M ® 7Lr 

& [a] Li 

We give outline of the proof that ( has the properties stated in Lemma 12.241 

The first part follows from a similar discussion to that in the proof that d o d = 0. Take a 
generator [r] G GFj_2/3^-iiYi). Then we have a cut-down moduh space Mj^{p,T; Li\Lq) with 
dimension 1. Counting the number of the end of this moduli space, we get 

5l o C = C o "^o. 

To prove the second part, take other metric G' and sections s^(p, a) over X. Then we have 
another map (' : GFF^:{Yq; 7) — CFF^,{Yi, ; 7'). We will construct a chain homotopy 

H : CFF,(Fo;7) ^ 7') 

such that 

(25) C -C = H odo + dioH. 

Take a path {G'*}o<s<i of metrics on X from G to G', and put 

M{p,a) := U M^ ^.(p,a) x {s}. 

0<s<l 
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Then M(p, a) is smooth for generic paths of metrics. We can define hne bundles Ci (p, a) over 
M(p, a) as usual. As in Subsection 12. 3[ for p, a with dim M(p, cr) < 8, we can construct sections 
s;(p, a) such that s(p, a) are compatible with gluing maps, {s;(p, a)}; satisfy the transversality 
conditions as in Proposition 12.81 and 

For < /3o < /3i < c?, -^^0 C Li C {1, . . . , (i} with |Lo| = /So, |-^i| = Pi and generators 
[p] G CFj_2^o(lo), [cr] G CFj_2/3i+i(yi), we have a moduli space M{p,a) with dimension 
2(/3i — /3o). Cutting down M{p,a) by the sections, we have the finite set M{p,a; Li\Lq) = 
M{p, cr) n flzGLiXLo -'^^^^ locus of si{p, 0"). We put 

< H{[p] ® 7lJ, [a] ® >:= #M(p, a; Li\Lo). 

These matrix elements give H : CFFj{Yo]'y) — )■ CFFj+i(Fi; 7') as usual. We can show the 
chain homotopy condition ( l25l) by counting the number of the ends of the cut-down moduli 
spaces M{p,T] Li\Lq) for generators [r] G CFj_2/3i(yi). Thus we obtain the second part of 
Lemma I2.24[ 

The proof of the third part is essentially same as the proof of the gluing formula for Donaldson 
invariants which will be given in the next subsection, and we omit the proof here. 

We give a remark on the dependence of HFF^,{Y;'~f) on 7. It seems that HFF^,{Y;'j) 
depends only on the homology classes [7/] G Hi{Y; Z). If 7/ and 7^ are homologous, then we 
have oriented, compact, surfaces in y x [0,1] with boundary 7z]j7;. As above we can 
construct a linear map 

Cs:CFF,(y;7)^CFF,(r;7') 

where S = (Si,...,Sd). To prove that induces an isomorphism from HFF^{Y]'j) to 
HFF^lY]^'), it is sufficient to show that the map has the properties as in Lemma [2.241 
The most difficult part of the proof is to prove that the induced map between the Fukaya-Floer 
homology groups is independent of S. 

Suppose that we have another bordism SJ from 7^ to 7^. We would like to show that the 
two maps between the Fukaya-Floer homology groups induced by Cs and Cs' are the same. A 
natural way to prove this is to use an isomorphism between (p, a) and Cf.,^ (p, a) which is 
compatible with the gluing maps in an appropriate sense, if the isomorphism exists. Hence the 
problem reduces to the existence of the isomorphism. This can be regarded as a generalization 
of the fact that the index of a family of elliptic differential operators on a closed manifold 
depends only on the bordism class. 

We can take a compact, oriented manifold W with corner, whose boundary is 

(26) SiUS;u(7zX [0,l])U(7;x [0,1]) 

as follows. Since [7;] is equal to [7^'] in Hi{Y; Z), there is a complex hne bundle L over Y and 
sections s, s' of L with s~"'^(0) = 7;, s~^(0) = 7^. Moreover there are sections s, s' of the 
complex line bundle L x [0, 1] over Y x [0, 1] with 

S|yx{o} = s, s\yx{i} = s', s'^{0) = S;, 

5'|yx{o} = s, 5'|yx{i} = s', s'^{0) = S;. 

Take a section s of the line bundle L x [0, 1] x [0, 1] over Y x [0, 1] x [0, 1] with 

-slyx^O.llxjO} = s|yx[0,l]x{l} = 5', s|yx{o}x{M} = S, s\yx{1}x{u} = s' ('UG[0, 1]). 
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For a generic s, the zero locus W of s is a compact, oriented manifold whose boundary is fl2^ . 
Attaching 7; x M>o x [0, 1], 7^' x R>o x [0, 1] to W, we get a "bordism" W from ti to SJ, i.e. 
is an oriented manifold with boundary S/ ]J S^. 

If we mimic the proof of the invariance of the index of operators over closed manifolds with 
respect to bordism, it may be possible to prove that there is an isomorphism from (p, cr) to 
Cf., [p, a) which is compatible with the gluing maps. However, we do not discuss this in this 
paper and we just state it conjecture. 

Conjecture 2.25. In the above notations, the bordism W between and gives a natural 
isomorphism between £g (p, a) and Cj., (p, cr) which are compatible with the gluing maps. 

If this is true, HFF^(Y; 7) depends only on the homology classes [7/] G Hi(Y] Z). 

2.7. Relative Donaldson invariants. In this subsection, we will see that the Fukaya-Floer 
homology groups allow us to extend the Donaldson invariants for closed 4-manifolds to 4- 
manifolds with boundary. 

Let F be a closed, oriented 3- manifold and 7 = {7z}f=i be a set of loops in Y, where < d < 

3. For compact, oriented 4-manifolds Xq with boundary Y, we write Xq for XqU (Y x ]R>o). 
We also denote E^U (7/ x M>o) by S/ for surfaces E; embedded in Xq with boundary 7^. Assume 
that Xq is simply connected and 6+ > 1. Moreover suppose that we have a [/(2)-bundle Pq 
on Xq such that the restriction Q = Po\y satisfies Hypothesis 12. 1[ Take a metric ^0 on Xq 
such that the restriction of gotoYx R>o is equal to gy + dt'^ for some metric gy on Y. Using 
families of twisted B operators over E;, we can define complex line bundles Cf^^p) over the 
moduli space M-^^{p) of instantons on the extension Pq of Pq to Xq with limit p. As before, 
we can show that for p with dimMj5^^(p) < 8 there are sections s;(p) of the line bundles with 
properties similar to those in Proposition 12.81 

Let Pq be the fixed flat connection used to define dy. For simplicity of notations, we suppose 
that 

ind Da{pq) = mod 8, 

where A[pq) is a connection on Pq with limit po. Then we have 

dim M^^ (p) = -6y ( [p] ) mod 8. 

Under these hypotheses, we define an element ipxo = i^Xg ^ CFFq{Y; 7) as follows. Let (3 be an 
integer with < (3 < d and take L C {1, . . . ,d} with \L\ = (3 and a generator [p] G CF_2/3(y). 
Then we get a number 

<V'Xo,[p]®7L >:= #M^„(P; L). 
Here Mj^^{p;L) is the 0-dimensional cut-down moduh space by the sections {si{p)}i^L- We 
define ipxo by 

:= 5^ 5^ 5^ < ^Xo, [p] ®1L> [p] ®lLe CFFq{Y; 7). 

/3 L [p] 

The following proposition follows from a combination of a discussion of the case when Y is 
a homology 3-sphere and techniques we have developed in this paper. 

Proposition 2.26. The chain ipxo 0, cycle, and the class \l/xo = ^Xol^i; • • • ; ^d) £ HFFq{Y] 7) 
represented by ipxo ^■5 independent of the metric and the sections. 

Remark 2.27. It seems that \l/xo(^i) • • • ? ^d) depend only on the homology classes [Ei], . . . , [E^] G 
H2{Xq, Y; Z). This is true if an analogy of Conjecture 12.251 holds for the line bundles Cf'^{p). 
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2.8. Gluing formula. We consider a situation where a closed, oriented 4-manifold X is cut 
into two parts Xq, Xi along a closed 3-manifold Y. Let P be a [/(2)-bundle over X such that 
the restriction Q = P\y satisfies Hypothesis 12.11 and dim Mp = 2d for some integer d with 
< d < 3. We suppose that Xo,Xi are simply connected and 6+ of Xq,Xi are larger than 
1. Let El, . . . , be embedded surfaces in X such that the intersections 7^ := F fl S/ are 
diffeomorphic to S^. We denote Xq fl S;, Xi fl by T,[, T,['. Then we have relative invariants 

^XoiK, HFFoiY; 7), v[/^, (S'/, . . . , S'j) G HFF,a-,{Y- 7). 

We will express the invariant \E'x([Si], . . . , [S^]) of the closed manifold X in terms of the 
relative invariants Xq-, ^ Xi- To do this, we define a pairing 

< , >: HFF,{Y;j) ® HFF.j^sMY;!) ^ Q 

as follows. 

For fiat connections p, we have 

Sy{[p]) = -6Y{[p])-3 mods. 

Hence CFj{Y) and CF_j_3(F) are dual to each other. We define a pairing 

< , >: CFF,{Y;2) ® CFF_,-_3+2d(l^; 7) Q 

by setting [p](8>7l<: G C*P-j-3+2/3(?) ®Q < Tl^^ > as the dual element of [p](8>7l G C'Pj-2/3(^)® 
Q < 7l >• Here L is a subset of {1, ... , d} with |L| = /3 and L'^ is the complement of L. It is 
easy to see the following. 

Lemma 2.28. < d{[p] O 7^), [a] O 7l/ >= ± < [p] ® 7^, 9([(t] ® 7/,/) > . 
Therefore we get the pairing 

< >:HFF,{Y;2)^HFF_j_3+2d{y;i)^Q 
on Fukaya-Floer homology groups. 

Theorem 2.29. In the above situation, we have 

vi/^([Si], . . . , [s,]) = 1 < M/xo(s;, . . . , s:,), vi/^,(s'/, . . . , s:;) > . 

To prove the gluing formula, we need sections of £f which are compatible with gluing maps 
as usual. Fix a Riemannian metric gy on Y and choose Riemannian metrics go,gi on Xq,Xi 
such that the restrictions to Y are equal to gy- For each T > 0, we define a manifold X**^^-* 
by X**^"^) = Xq U y X [0,T] U Xi. Then go,gi naturally induce a Riemannian metric g*^"^^ 
on X#(^). Fix a sequence — )■ 00 and we write X" for X'^^'^°'\ As in Subsection 12.31 we 
can construct sections sf of £f ^ — j- such that for each sequence [A"] G Mx« converging 
to . . . , [A^]) the values at converge to sz([A§°]) K ■ ■ ■ S si{[A^]) in an 

appropriate sense. By dimension counting, we can show that if [A°'] lie in the intersection 

d 

1=1 

then r = 1 and {[A^], [A^]) E M^^{p; L) x M^^(p; L^). Here [p] G CF_2p{Y) with < /3 < d 
and L is a subset of {1, . . . with |L| = /3. The transversality conditions for the sections 
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imply that M^^^(p; L) and M-<^^{p] L'^) are finite sets. From an argument like that in the proof 
of ([7j), we can show there is a natural identification 

d d 

Mx. n fl = U U U ^Xo(P; L) X M^^(p; L^) 

1=1 (3=0 5y{[p])= \L\=P 

-2/3 mod 8 

for large a. Counting the number of elements of both sides with signs, we obtain the gluing 
formula. 

3. Floer homology for 2-torsion instanton invariants 

In this section, we consider a variant of Donaldson invariants for non-spin 4-manifolds 
X, which is a linear function 

: A'{X) Z2. 

Here A'{X) is the subspace of Q)d>oH2{X; Z)®°' generated by the set 

{ [Si]®---® [SJ I pi] G/f2(X;Z), [S;]-[S,]=0 mod 2}. 

This invariant is defined by a 2-torsion cohomology class ui of the moduli spaces. Originally 
this was defined for spin manifolds by Fintushel-Stern [9], and the author extended to non-spin 
4-manifolds in [13]. In this section, we construct a variant of Floer homology group and prove 
a gluing formula for . 

In [13], it was shown that is non-trivial for X = CPg#CP?#Cf2, where CPg and CP? 
are copies of CP^. More precisely 

(27) {-Ho + E,Hi- E) = 1 mod 2. 

Here i^o, Hi and E are the generators of H2{CFl; Z), if2(CP?; Z) and i/slCP^; Z) respectively. 
We can consider X = CPg#CP?#CP2 as a connected sum of Xq = CPg and Xi = CP?#CP2. 
Since the only flat connection over is the trivial one, we can deduce from ( 1271) that the 
trivial fiat connection has an important role in the gluing formula for in contrast to the 
case of usual Donaldson invariants. Note also that the homology class —Hi + E in ( 1271) is a 
sum of homology classes —Hi and E of Xq and Xi with self-intersection numbers odd. 

On the other hand, for closed, simply connected, non-spin 4-manifolds Xq,Xi with pos- 
itive, we can show the following vanishing theorem. For homology classes [S^] G H2{Xq\7j), 
[Sy G H2{Xi] Z) with self-intersection numbers even, 

(28) nWxi(Pi], - - - , [S.o], [S;], . . . , [S:,J) ^ mod 2. 

This follows from a standard dimension- counting argument. The formula (125]) implies that 
the key of the non- vanishing result (I27|) is that the homology class —Hi+Eis split into two 
homology classes of Xq^Xi with self-intersection numbers odd. 

In this section, we consider a situation where X has a decomposition X = Xq Uy Xi for 
a homology 3-sphere Y and a closed, oriented surface S in X is split into two surfaces with 
self-intersection numbers odd along Y. We will give a gluing formula for \I'^^([S]) in terms of 
differential-topological date about Xo,Xi. This situation is similar to that of Section [2] and 
we can apply the method developed in Section [2J The main difference is that a f/(2)-bundle 
over a homology 3-sphere has the trivial flat connection, which is reducible, and we need to 
pay attention to the effect of the trivial fiat connection on the gluing formula. 
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3.1. 2-torsion instanton invariants for closed manifolds. We will summarize the con- 
struction of 2-torsion instanton invariants for closed non-spin 4-manifolds which is given in 
US]. See also [S]. 

Let X be a closed, oriented, simply connected, non-spin 4-manifold with b'^ > 1. Take a 
f/(2)-bundle P over X with W2{P) = W2{X). For [S] G H2{X;7j), we have the complex line 
bundle 

We can show that if [S] ■ [E] = mod 2 then the center {±1} of SU{2) acts trivially on the 
line bundle. Hence we get a line bundle 

£s = ^s/^0(3) ^ . 

Let c be a spin-c structure of X. For each connections A on P, we have the twisted Dirac 
operator 

Here E is the rank two complex vector bundle associated with P and are the spinor bundles. 
If Ci(detc) = — Ci(P), we have a "real part" of the Dirac operators: 

{Pa)r : r((S+ ® E)m) T{{S- ® P)m). 

(See [T].) The family of real operators {{^A)R}[A]eBp gi'v^s a real hne bundle 

We also suppose that C2(P) = mod 2. Then we can see that the center {±1} of SU{2) acts 
trivially on A, and we obtain a real line bundle 

A = A/S0{3) A B*p. 

We define Ui G H\B*p; Za) to be Wi{A). 

When b'^{X) is even, the virtual dimension of the moduli space Mp is odd. We can write 
dim Mp = 2d + 1 for some integer d. Assume d > 0. Then we define 

• • • , Pd]) = " < Ml U Ci(£eJ U ■ • • U Ci(£eJ, [Mp] > " G Z2. 

Here [Si], . . . , [SJ G H2{X;'Z) with [Sj] ■ [Sj] = mod 2. In general Mp is not compact, 
however we can define the pairing as follows. Let ss. : B^. — )■ be sections and Vj]. be the 
zero loci. We can prove the following lemma by a standard dimension counting argument. 

Lemma 3.1. Assume that the dimension of Mp is 2d + r for some d > and < r < 3. 
Then the intersection Mp fl fl ■ • • fl Vs^ is compact and smooth for generic sections Sy... 

When dim Mp is 2(i -|- 1, the intersection 

Mp n n ■ ■ ■ n v^^ 

is a compact, smooth manifold of dimension 1. Precisely the invariant \l'^^([Si], . . . , [S^]) is 
defined to be 

< Ml, [Mp n n ■ ■ • n i^sj >e Za. 

We can show that \&^^([Ei], . . . , [S^^]) is independent of the metric on X and sections of £3^5 
and hence it is a differential-topological invariant of X. 
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Remark 3.2. When we define the invariant \l/^^([Ei], . . . , [E^]), we use the pull-back r^-{sj].) 
of the section ss. over B^.. Here r^^ is the map Mp — )■ B^. defined by restricting connections 
to Sj. However Lemma 13.11 shows that when the dimension of Mp is 2(i + 1 with d > 
Mp n n ■ ■ • n Vs^ is a compact, smooth manifold of dimension 3. Hence we need not to 
use the pull-back ^^^(ssi)- We can use any section of r^-^{Cj]-^) which is transverse to the zero 
section. 

3.2. Line bundles. In this subsections, we introduce line bundles over Byxr{Pj ct) which are 
defined by families of Dirac operators over y x M and twisted d operators over surfaces in 
y X M. These are used to define a variant of Floer homology groups for 2-torsion instanton 
invariants. 

Let y be a homology 3-sphere and Q = Y x U(2) he the trivial t/(2)-bundle over Y. Fix 
a connection a^et on Qdet = Yx f/(l). As before, connections on Q and 7i*Q are compatible 
with Odet, unless explicitly stated otherwise. 

We assume the following. 

Hypothesis 3.3. All fiat connections on Q are non-degenerate. 

For irreducible fiat connections p,a, we define Byxr{p,<^), ^YxuiPj'^) before. We have 
just one gauge equivalence class of projectively fiat connections which are not irreducible. It 
is represented by a connection 6o which induce the trivial connection on the adjoint bundle 
of Q. Fix a smooth map gi : Y ^ SU{2) with degree 1 and put Oa '■= Qiido) for a G Z. We 
define i3yxM(^a, cr), ^Yy.R{p,Ga) as usual. Let Fp be the stabilizer of p in Qq. Then we have a 
natural action of Vp x V„ on Byy^^i^p, o"), and Byy^^XP^ ^) — ^YxR^Py ^)/^p ^ ^o-. Note that the 
action of the subgroup {±(1,1)} C Fp x F^. on i3yxjj(p, a) is trivial. 

For flat connections p, a, we have real line bundles 

A(p,o-) ^ BY^^{p,a) 

induced by families of the real part of Dirac operators. Since the action of {±(1, 1)} C Fp x Fo- 
on the line bundle is not trivial in general, the hne bundle may not descend to By^^ip, cr). To 
avoid this problem, we will introduce real line bundles ^{9a, p) and M(cr, 6'6) over Byxuip,'^) 
for irreducible flat connections p,a and a,b ^ Choose connections A{9a,p), A{(T,9h) on 
7r*Q ^ y X M such that 

A{e.,p) = l^^ onyx (-00,-1), 
^ '^^ [ p on y X (1, oo) . 

^ ^ [6b on y X (1, oo) . 
Then we have the Dirac operators 

^Aie.,p) : L'/'^''\S+ ^E)^ L'/-''^\S- ® E), 

^Ai.M ■■ Ll'^~^'^\S^ ®E)^ Lf~'^^\s- ® E). 

We denote the numerical index of these operators by Ind^^ 'pA{ea,p)^ Ind""*" pAiufif,)- Define 
M.(6'a,p) to be trivial line bundle with action of Fp = Z2 of wight Ind^^ pA{ea,p)- Similarly 
R.(cr, 9b) is the trivial real line bundle with Fo- action of wight Ind"'*' ^yi(cr,6»b)- Here we put 

A„fe(p, a) := R{Ga, p) ® Hp. ^) ® 0b)- 
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The wight of the action of {±(1, 1)} C x r^. is 

Ind-+ ?SA(e„,p) + Ind-+ + Ind-+ '^A{aA) = Ind^^ ^Aie^A)- 

Here [A] G Byy^^iPjCr). If Ind^'^ /^A{9a,eb) even, the hne bundle Aaft(p, cr) descends to 
^yxR(P>^)- 

Lemma 3.4. Ind"+ ^A{eaA) = -{b-a). 

Proof. Let Pb-a be a [/(2)-bundlc over Y X with ci(P6_„) = 0, C2{Pb-a) ^b-a. It follows 
from the additivity of index and the index formula that 

Ind-+ ^AiOaA) = ch{Pb-a)A{Y X S')/[Y x S']. 

Since ch{Pb-a) —2 — C2{Pb-a) and A(F x 5^) = 1, we have 

Ind-+ '^Aie.M = -< C2{Pb-a), [Y X S'] >= -{b - a). 

□ 

For a, 6 e Z with b — a = mod 2, we get a real hne bundle 

Kb{p,(^) ■■= Aab{p,(^)/S0{3) — > BY^f.{p,a). 

Let a, b, a', b' be integers with a = b = a' = b' mod 2. Then we have a natural isomorphism 
from M.(6'a', Oa)^Aab{p, a)^'R{9b, Ob') to A(j/6/(p, a) and we can see that this isomorphism induces 
an isomorphism from Aaf,(p, cr) to Aa'6'(p, cr). Therefore we obtain: 

Lemma 3.5. If a = b = a' = b' mod 2, Aab{p, o") and A^'b^ip, c) are isomorphic to each other. 

Definition 3.6. For irreducible flat connections p.a and a G {0,1}, we write A*^")(p, cr) for 
Aaa(p, cr). Moreover we define u^-^^ = u{\p,a) by 

Let 7 = 5"^ be a loop in Y . Then we have the determinant line bundle 

£r ^yxR(P><^)- 

Here F = 7 x R. This line bundle does not descend to By^^[p.^ a) in general, we can however 
apply the technique used to define Aab{p, o"). 

Let 9^fi be the restriction of 9o to 7. Choose a smooth map gf^ : 7 — )■ t/(2) such that the 
homotopy class [g^] is the generator of 7ri([/(2)) = Z. Then for a e Z we put 

Note that these connections are not compatible with the fixed connection ajct on Qdct- For 
a,b E 1^ and irreducible flat connections p, a, we take connections A{9j^a,p), ^{cr, on n*Q 
such that 



Ai9^,a,p) 

A{a, 9^,b) 



9^^a on y X (— 00, — 1), 

p on y X (1, 00), 

a on y X (—00, —1), 

O^^b on y X (1, 00). 
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(These connections are not compatible Odet either.) Let C{9^^a, p) and C{a,9.y^b) be the trivial 
complex line bundles over Byy^jg^ip, cr) with Fp-action of weight Ind""*" dA(9^^^^,p) and Fcr-action 
of wight Ind""*" 9yi(o-,e^ t) respectively. Put 

^r,ab{p^ ^) •= C(^7,«' P) ® ^rip, cr) (g) C(o-, e^^b) — > ^YxuiP^ cr)- 
Then Tp x To- naturally acts on Cr,abiP^ ^^e weight of {±(1, 1)} is 

Ind"+ <9A(e^,,,p) + Ind~+ dA(p,a) + Ind~+ 9^(<T,e^,t,) = Ind~+ 
If this number is even, the line bundle £r,a6(p, o") descends to Byxr{P:<^)- 
Lemma 3.7. Ind"+ <9A(e^,„,e^ ,) = b-a. 

Proof. Let P-y^h-a be a f/(2) bundle over 7 x S*^ with ci(P^,b„a) = b — a. From the additivity 
of index and the index formula, we have 

Ind-+a^(e,,,,e,,,) = c/i(P,,,_ji(7 X ^i)/[7 X S'] 
= (2 + ci(P^,,_J)/[7x5i] 
= 6 — a. 

□ 

This lemma implies that for a,b E 1j with a = b mod 2 we can define 

^r,ab{p,(^) ■■= CY,ab{p,(y)/^p X F^ ^ B*Y^^Xp,a). 

We can also show that for a = b = a' = b' mod 2 we have a natural isomorphism be- 
tween Cr,ab{p,cr) and C-p^a'b'ip,'^)- We get two bundles Cf\p,a) := £r,oo(P)Cr), £p^(p, a) : = 
£r,ii(P5 cr). We will use only C^\p, cr) to define a variant of Floer homology. 

3.3. The complex. Using the line bundles introduced in the previous subsection, we construct 
homology groups which allow us to extend 2-torsion instanton invariants to 4-manifolds with 
boundary. The idea of the construction is fundamentally the same as Fukaya-Floer homology 
groups in Section O The main difference is the existence of a reducible flat connection. 

Fix a G {0, 1} and take sections 

SA = SA(p,a) : M°,jj(p,(t) — > A(")(p,(t) 

sc = sc{p,(t) : M^^^{p,a) — > C'^\p,a). 

We denote the zero sets by V\, V\. For sets 

Lo = 0, Li = {A}, L2 = {C}, L3 = {A,£}, 

we define 

M^xr(p, ^0) := M° ^jj(p, a), M^^^{p, a; L,) := ^^(p, a) f] V^a, 
M° ^j,(p, a; L2) := M° ,j,(p, a) H Vc, M^^^{p, a; L3) := M° ^^(p, a) n ^a H l^^. 

Lemma 3.8. Let p, a 6e (possibly reducible) flat connections on a U{2)-bundle Q over a homol- 
ogy 3-sphere Y with dim My ^jg(p, cr) < 3. Take a sequence {[A°']}a in Myy^^iPyCr) converging 
to i[Af], [A^]) with r > 1. Here [A°°] E M^^^{p{i - l),p(i)). Then p{i) are irreducible 
for i = 1, . . . ,r — 1. 
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Proof. The dimension of dim Myxm.{p, c) is given by 

dim Myxr{p, cr) = Ind"^"^ Dac = Ind""*" Da^ - dim 
where is the Lie algebra of Tp. The additivity of index imphes that 

r r 

Ind-+ Da^ = J2^nd-+DAoo = ^ {dimMyMi - 1) , p(i}) + dim H°^^,_^^) . 

i=l i=l 

Hence we have 

r r—1 

dimMyxM(p, cr) = ^ dim My xR(p(i - + dim •) . 

i=i i=i 

Assume that p(io) is reducible for some io with 1 < io ^ ''^ ~ 1- Then p(io) is the trivial flat 
connection and the dimension of is 3, and dim My xM(p(i — l),p{i)) > 1. Hence 

4 > dim MyxR(p, cr) > r + 3 > 4. 

This is a contradiction. 

□ 

This lemma means that we can apply the method in the previous section to showing that we 
can take admissible sections of the line bundles A(")(p, cr), jC^\p, cr) when dim My cr) < 3. 
Thus we obtain the following. 

Lemma 3.9. Let p,a be irreducible flat connections on Q with dim My (p, cr) < 3. Then we 



can take sections sa, S£ of hS-"'\p,a), {p,a) with the following propert 



ICS. 



• Let {[A'^]}a be a sequence m M^^^{p, a) with [A"] {[Af], [A^]). Then we have 

sa(K]) ^ s^{[Ar]) K • ■ ■ K s^{[A^]), 
sc{[A^]) sc{[Ar]) m---m sc{[A^]). 

• Let (3 be an integer with < (3 <3. If dim Myy^^ip, cr) < (3, then 

M° xm(p, <j; h) = 0. 

• //"dim My cr) — (3, then the intersection My^^^ip, cr; Lp) is transverse and compact. 
Hence My^]g(p, cr; L^) is a finite set. 

In this section, we refer to CFi{Y) as the Z2-vector space generated by gauge equivalence 
classes [p] of irreducible flat connections with 5y([p]) = i mod 8. Using admissible sections of 
A(")(p, cr), jC^\p, cr), we define the complex as follows. 

Definition 3.10. 

(1) 



Cf{Y) := { 



CF8„_/3(r) I © < > if J = 8n, n = a + 1 mod 2 , 



./3=0 

3 



^CFj^p{Y) otherwise. 

13=0 

(2) We define d^"^ : Cf{Y) d!.\(y) as follows. 
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(a) For integers /3i,/32 with < (3i < (32 < 3 and generators [p] G CFj_j3^(Y), [a] G 
CFj_/32-i(V'), we put 

< d^^Hlp]) W] >■= I ^^°xi8(^'^'^/^2\^/3i) mod 2 if L^^ C L^j,, 
^ ^ ' ]^ otherwise. 

(b) For [p] G CF8„+i(F) with n = a + 1 mod 2, we put 

< d'^''\[p]),9^ >:= #M°,i,(p,e„) mod 2. 

(c) For [(t] G CFsn^4^(Y) with = a + 1 mod 2, we put 

< 9(")(^„,), [a] >= #M°,K(^„,a) mod 2. 
We claim that d^""^ o 5'-"^ is identically zero. 

Lemma 3.11. d^"^ o d^"^ = 0. 

We must show that for generators [p] G Cj"^(F), [cr] G C'j!l2(F) 

(29) < o 9(")([p]), [a] >= mod 2. 

We split the proof into four cases. 

(1) Flat connections p,a are irreducible and [p] G CFj_^^(F), [a] G CFj_;32-2(^) for some 
integers /32 with < /^a - /3i < 2. 

(2) [p] G CF8„4_2(^), ci = On for some n G Z with n = a + 1 mod 2. 

(3) p = 6'„, [a] G CF8„_5(F) for some n G Z with = a + 1 mod 2. 

(4) [p] G [a] G CF,_5(r). 

To prove (!29|) . we need to describe the ends of Myxjj(p, a) as in the previous section. In the 
cases (1), (2), (3), the dimension of the moduh space My^]g(p, a) is less than 4. Lemma [3^81 
means that (l29l) follows from the same discussion in the previous section. Therefore we need 
to consider only the case (4). 

Lemma 3.12. Take generators [p] G CFj{Y), [a] G CFj_5(F). We can take sections sa, sc of 
the line bundles A^"-^{p,a), C^\p,a) with the following property. Let {[A"]} a be a sequence in 
MY^^{p,cr) converging to some ([^4^"], . . . , [AJ^]). Here [A°°] G My^]g(p(z — l),p(z)). Suppose 
that p{i) are irreducible for z = l,...,r — 1. Then 

sa([a"]) .A([Ar]) K ■ ■ ■ K sa([4°°]), 

We can show this by using a partition of unity on the moduli space as Proposition 12. 8[ The 
following lemma follows from the usual dimension-counting argument. 

Lemma 3.13. Take generators [p] G CFj{Y), [a] G CFj^^iY). Let {[A'^]\a be a sequence in 
My^]fj(p, 0"; L3) converging to ([Af^], . . . , [A^]) with r > 1. Then we have either 
• r = 2 and 

[AT] e M°,i,(p,p(l);L^), [A^] G M°,^(p(l),a;L3\L^) 

for some integer (i with < /3 < 3 and an irreducible flat connection p(l) with [p(l)] G 
CF,_p_^{Y), or 
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• r = 2, p(l) = On for some n and 

dim M°^K(P>^n) = dimM°^R(^„,cT) = 0. 

This lemma means that the reducible flat connections On do not appear in the description of 
the end of My^]g(p, cr) if [p] ^ CF^n+iiX) for any n. Therefore (l29l) follows from a discussion 
like that in the previous section in this case. 

From now on, we assume that [p] G CF^n+iiX)i M £ CF^n-AiX)- To describe the end of 
the cut-down moduli space My^]g(p, a; L^), we put 

3 

Nirr := U U ^vxr(p,p(1);^/3) X M° (p( 1) , (T; ^L^) , 

/3=0 [p(l)] 

iV.ed := M°^k(p,^„) X M°^j,(5„,a), 

• -^irr y y -^red ■ 

Here [p(l)] runs over generators of CF8„+i_/3(F). For each [A\ = {[Ai], [A2]) G Ni^r, we have 
a gluing map 

C/fA] : U[A,] X (To, 00) X f/[A2] — > M^^^{p,a) 

for some precompact open neighborhoods U[Ai],U[A2] of [^1]) [^2] and Tq > 0, and for [A\ = 
{[Ai], [A2]) G Nred, we have a gluing map 

G/[A] : (To, 00) X ^0(3) ,k(p, 

For Ti > To, we write G/[a],t>Ti for the restriction GI[a] to the region where T > Ti. Here we 
put 

M' = M^^ := M°,K(p,or;L3) \ |J ImG/^,r>T,. 

Lemma 13.131 means that M' is compact. For irreducible fiat connections p(l), the sections 
of the line bundles over MyxR(p, p(l)), My^]g(p(l), cr) satisfy the transversality conditions in 
Lemma [3.91 Hence from a discussion like that in the proof of ([7]), we have 

(30) M'n [j ImG/[A],Ti-l<T<Ti = N,rr X (0, 1] 

\A]€Nirr 

for large Ti. Perturbing the sections sa, S£ outside the end (l30l) of M', we get a 1-dimensional, 
smooth, compact manifold with boundary 

N,rr u U (50(3)[jnVAnv,:). 

Here S'0(3)[j is the image of 50(3) x {Ti} by We write S'[a] for the intersection 

5*0 (3) [J n Va n V/;. Then we obtain 

#iV,,,+ J] #5[A] = mod 2. 

We must show 

. . // c _ / 1 mod 2 if n = a + 1 mod 2, 

^"^^ ^"^[^1 " \ mod 2 otherwise 

to get f l29p . This follows from the following. 
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Lemma 3.14. The restriction £p ^(p, cr)\so{3)i^ is non-trivial, and the restriction A'^^^p, a")|5o(3) 
is non-trivial if and only if n = a + 1 mod 2. 

Proof. Let Xi,X2 be two copies of F x M. And we write Xi(T), X2(T) for Y x (-oo,2T), 
Y X (-2T, oo). Gluing Xi(T), X2(T) through the identification 

Y X (T, 2r) ^ Fx (-2T, -T) 
^ (l/,t-3T), 

we get a manifold X*^'^\ 

Let be the rank-two complex vector bundle over Y associated with Q. Take a trivialization 
(j) of E such that 6n is trivial under 0. We write Ei for 7r*i? and 0j for 7r*0. Here vTj are the 
projections from X^iT) to F. For -u G SU(2) we define a vector bundle -E('u) by 

Gluing two instantons ([^i], [^2]) ^ A^red, we get an instanton [A('u)] on E{u), and we have 
the determinant line bundle {^r\A{u)]}uesu{2) over SU{2). We have an action of {±1} on this 
bundle defined by the following diagram: 

(C-^Aiu)] (C(^^^,i,p)®(£)[A,])® ((/:)[A.]®C(a,^^^,l)) 

(-l)"il+'"2(g) 



mi = Ind mddA(e-,^i,p), ^2 = Ind Oa^. 
"''^ H-^otient oi \^r,iAiu)]. 

)[A] 



Here 

The restriction C^\p,a)\so{3)iA] is the quotient of {-^r [yi(u)]}uGSC/(2) by this action. The nec- 
essary and sufficient condition for Cy\p,o')\so{3)ia] non-trivial is that nii + m2 is odd. 
From the additivity of the index, we have 

mi + m2 = Ind"'+ <9A(e^,i,0„|^). 

Here A(^^ i,6'„|^) is the connection on F = 7 x M with limits 6^^i, 6'„|^. Let gi be the fixed 
map from Y to SU{2) with degree 1. The class [gil-y] represented by the restriction of gi to 7 
is trivial in 7ri([/(2)). Hence we have 

mi + m2 = Ind"'+9A{e^,i,e^,o) = "1 

by Lemma [STl Therefore the restriction C^\p,a)\so{3)fA] non-trivial. 

Similarly the necessary and sufficient condition for A^°-\p, o")|5o(3)[ai non-trivial is that 

Ind"'+ f>A{ea,en) = 1 mod 2. 
Lemma 13.41 implies that this condition is equivalent to 

n = a -\- 1 mod 2. 

□ 

Definition 3.15. /i"^(F;7) := H,{Ci''\Y),d^''^). 
As in Subsection 12. 6^ we can show the following. 
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Proposition 3.16. I^"'\Y;'y) is independent of the metric on Y and the sections of A^"'"' {p, a) , 

Remark 3.17. We assumed that F is a homology 3-sphere, and it is sufficient to take a very 
small loop 7 which is trivial in vri(y) when we calculate invariants using the gluing formula. 
However we have considered a general loop 7 which may not be trivial in 7ri{Y). One of 
the reason is that we would like to prepare techniques to extend the construction to more 
general 3-manifolds. Another reason is that the dependence of /*(y;7) on 7 is not clear and 
investigating the dependence may be an interesting problem. 

We show the subtlety of the dependence of I^:{Y;'y) on 7. Let 7 be a very small loop in 
Y whose homotopy class is trivial in ni{Y). Then we can see that the complex line bundles 
C^\p,a) are trivial and it seems that the components of the boundary map d^""^ relevant to 
7 are trivial. However this is not true in general. If we would like to show the components 
are trivial, we need to show that there are trivializations of the line bundles L^\p,a) over 
MyxR(p, c") which are compatible with the gluing maps. Let p, a be irreducible flat connections 
over Y with dim M^^^{p, a) = 0. We can see that the kernel and cokernel of the twisted 
d are trivial for the small loop 7 and hence we have a canonical trivialization of the line 
bundle £r(p, cr) over My^^ip, cr). However this triviahzation does not canonically descends to 

Myxjg(p, a). The line bundle C^\p, cr) is deflned to be the quotient of C(6'^^a, p) ® jC.r{p, o") ® 
C((T, O-y^a) by the action of Z2 x Z2, and the action of Z2 x Z2 is not trivial in general. Therefore 
we do not get a canonical trivialization of C^\p, cr). This means that for flat connections p', a' 
with dim My xr(p', cr') equal to 1 or 2, there are no trivializations of the line bundles C^\p', a') 
over the end of My^jg(p', a'). Hence in general, there are no trivializations of the line bundles 
over moduli spaces which are compatible with the gluing maps. 

3.4. Relative invariants. Let X be a closed, oriented, simply connected, non-spin 4-manifold 
with b^{X) positive and even. We consider a situation where we have a decomposition X = 
XoUyXi. Here Y is an oriented homology 3-sphere and Xq, Xi are compact, simply connected, 
non-spin 4-manifold with boundary Y, Y and 6"'"(Xo), 6^(Xi) > 1. We consider invariants 
^xp(^)' where P is a t/(2)-bundle over X with W2{P) = W2{X) and dim Mp = 3 and z e 
H2{X; Z) with z ■ z = mod 2. The purpose is to write "^^{z) in terms of data from Xq, Xi 
as in the case of the usual Donaldson invariants. 

We flx a trivialization ipQ of Q := P\y and we write ^0 for the trivial connection. Choose a 
smooth map gi from Y to SU{2) of degree 1. For n G Z, we denote gi{6o) by ^„ as before. 
For a sequence — 00, put X" = Xq U (Y x [0, T"]) U Xi. We can easily show: 

Lemma 3.18. Take [A°'] G Mx-^^p and assume that [A"-] converges to ([^[f], . . . , [^J^])- Here 
[Af] G MyxM(p(^ ~ 1))P(0)- Then we have either 

• p{i) are irreducible for z = 1, . . . , r — 1, or 

• r = 1 and p(l) = On for some G Z, moreover dimMj5^^(6'„) = dim M^j^^ (6'„,) = 0. 

This lemma means that the construction of the gluing formula for \E'^^(z) is the same as 
that in Subsection 12.81 if dim M^^(6'„) 7^ for all n G Z. Hence we suppose the following. 

Hypothesis 3.19. There is an integer hq with dimMj^^(^^„,)) = 0. 

We will deflne relative invariants for Xq under this assumption. Put Pq •= P\xo and let 
7 = 5*^ be a loop in Y and So be an oriented surface embedded in Xq with boundary 7. Using 



40 



HIROFUMI SASAHIRA 



Dirac operators associated with a spin-c structure Cq over Xq with ci(detco) = — Ci(Po) and 
twisted d operators over Sq) we have hue bundles 

A(p)A%(p), 4^^s^^(p). 

For irreducible flat connections p and a G {0, 1} we put 

AW := A(p) ® M(p,0,), £g(p) := 4^(p) ® C(p,0,,i)- 

Here ^^^i is defined as the previous subsection. Then Tp = Z2 naturally acts on these bundles. 

Lemma 3.20. Let ipo be the fixed trivialization of Q and put '■= di'Po- When C2{Po, (fa) = 
mod 2, A^"-\p) descends to B*^^{p). Let zq e H2{Xo, Y; Z) = H2{Xo; Z) be the class represented 

by TiQ. If Zq ■ Zq = 1 uiod 2, the line bundle jC^^\p) also descends from B*^ (p) to B*^ (p). 

Proof. The wight of the T p action on PS'^\p) is Ind"^ 1^A(0a)- Hence it is sufficient to show that 
the index is even if C2(Po; V'a) is even. Choose a compact, spin 4-manifold X'q with boundary 
Y and we consider a manifold X' = Xq Uy Xq. Let P' be a ?7(2)-bundle over X' obtained by 
gluing Pq and the trivial bundle over Xq through ipa. Let c'q be a spin-c structure over Xq 
induced by a spin structure. Then we have a spin-c structure a' on X' induced by Cq, Cq and 
we have the Dirac operator 

^A' : r(^+ ® E') T{S^ ® E'). 

Here A' is a connection induced by A{6a) and the trivial connection 6x[^ on Px^ and E' is the 
complex vector bundle of rank two associated with P'. Since , is the direct sum of two 

copies of a Dirac operator, the index Ind~ Vg , is even. Thus we get 

Ind+ '^A(e^) = Ind+ ^A(ea) + Ind" ^e^, mod 2 
= Ind '^A' mod 2. 
Applying the index formula, we have 

Ind^A' = c/i(E')i(X')e^-i(detc')/[x'] 

= 2i(X')e5^^('i°*^')/[^'] + ^ci(P')(ci(det c') + c,{E'))/[X'] - C2{E')/[X']. 

Since A(X')e2^i('^<=*^'V[X'] is the index of a spin-c Dirac operator, it is an integer. Hence the 
first term is even. From ci(detc') = —ci{E'), the second term is zero. Therefore we obtain 

Ind+ ^Aie.) = -C2{E')/[X'] = c^iPo; (fa) mod 2. 

Thus the first part of Lemma follows. 

To show the second part, we need to show 

Ind"*" dA{e.y,i) = mod 2. 

Here A{9.y^i) is a connection on Xq with limit 6^^i. The additivity of index implies that 

Ind+ dA{e^,i) = Ind+ dA{e^^o) + Ind"+ dA{e^^oA,i)- 
Applying Lemma 13.71 to the second term in the right hand side, we get 

Ind+ <9A(e^,i) = Ind"^ dA{e^^o) + 1- 
Let (p^ be the restriction of (po to 7 and take an oriented, compact surface Sq with boundary 
7. Gluing PqIso and the trivial f/(2)-bundle Ps(, over Sq through ip^, we get a f/(2)-bundle P^/ 
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over S' = So Sq. Let 6' be the trivial connection on Pq. The index Ind dg' is even, since 
80' is the direct sum of two copies of the ^-operator. Thus we have 

Ind"^ dA(e-,^o) = Ind+ <9a(0^,o) + Ind" dg> mod 2 
= Ind (9^/ mod 2. 

Here A' is the connection over S' induced by A{6^^q) and 6'. From the fact that ci(Po) = ^2(-^o) 
mod 2 and ■ ^0 = 1 mod 2, we have 

Ind^A' = c/i(Ps')^(S')/P'] 
= (2 + ci(PsO)/[S'] 
=<ci(Ps'),[Sl > 

= < Ci(Po), Zo > 

= 1 mod 2. 

Therefore we obtain 

Ind+9A(e^_i) = mod 2. 

□ 

Definition 3.21. Let a G {0, 1} with C2(Po; <y5a) = mod 2. For irreducible flat connections 
p, we define 

A('^)(p) := A('^)(p)/rp A Bl^ip), 4l{p) := ^g/F, A i3^^(p). 
The following lemma follows from a dimension counting argument like the proof of Lemma 

m 

Lemma 3.22. Let p be a flat connection on Q with dimM^^(p) < 4 and {[A°']}a be a sequence 
in Mj^^^{p) which converges to some {[Af], . . . , [A'^]). Here [Af] G My^]jj(p(i — 1), p{i)) and 
p{r) = p. Then p{i) are irreducible for i = 0, . . . ,r — 1. 

This lemma implies that for p with dim Mj^^{p) < 4 we can take sections sa(p) : Mj^^^{p) — )■ 
A(")(p), sc{p) '■ Mj^^^{p) — > C^^\p) having properties like those in Proposition 12. 8[ Using 

admissible sections, we define ip"^^ G Cg^g{Y) as follows. Let /3 be an integer with < /3 < 3 
and take a generator [p] G CFgno-piX)- Then put 

<rx\,[p]>--=mxo(p-^Lp) mod 2, 

and we define 

/ ^^0(6*0) mod 2 if no = a + 1 mod 2, 
^^xo'f^no^- I otherwise. 

These numbers define a chain ip"^^ G Cg'^^{Y) as usual. As before, we have: 

Proposition 3.23. The chain tp^^^ is a cycle. Moreover the class '^'^^ = ["^Axq] ^ -^8nl(^!7) ^-^ 
independent of the metric on Xq and the admissible sections o/A^")(p), £^^''(p). 

Remark 3.24. As the case of relative Donaldson invariants "^Xq, if an analogy of Conjec- 
tur d2.25l holds then relative 2-torsion instanton invariants \E'^^^ is independent of the surface 
Eq representing the homology class zq. 
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3.5. Gluing formula. We begin witli tlie definition of a pairing bmce 
5y([p]) = — 5y([p]) — 3 mod 8, we liave a natural pairing CFj(Y) (g) CF_j_3(F) — )• Z2. For 

[6n] G Cg'^{Y) and [6-n] ^ C^l^{Y), we define < dn,0_n >= 1 mod 2. Then we get a pairing 

< , >: -^Z2. 

We can easily show 

Lemma 3.25. For [p] E C^°'\y) and [a] e CF[^^^^{Y), 

<d^'^\[p]),a>=<[p],d^'^\[a])>. 
This lemma means that the pairing induces a pairing 

<, >:/f (y;7)®/i';^(?;7)^Z2. 
The main statement in this section is the following. 

Theorem 3.26. Let X be a closed, oriented, simply connected ^-^o-nifold with positive 
and even. Assume that X has a decomposition X = Xq Uy Xi, where Y is a homology 3- 
sphere and Xq,Xi are compact, non-spin, simply connected 4-'m(inifolds with boundary Y, Y. 
Take a loop 'j in Y and compact, oriented surfaces Sq, Si in Xq, Xi with boundary 7. // the 
self-intersection numbers of Zi := [Sj] G H2{Xi,Y;Z) = H2{Xi;Z) are odd, then we have 

where z = Zq -\- Zi E H2{X; Z). 

We can generalize this formula to the case when the dimension of Mx,p is more than 3. 
Assume that the dimension of Mx,p is 2((io + (ii) + 3 for some ^0,(^1 G Z>o. For xi, . . . , G 
H2{Xq; Z), yi, . . . ,yd^ G H2{Xi; Z) with Xi ■ Xi = yj ■ yj = mod 2, we can define relative 
invariants 

^"-jIM, . . . , So) G li^\Y; 7), n\iyi, . . . , Si) G li'^^Y; 7). 

Here Sj are surfaces in Xi as in Theorem I3.26[ Then we have 

'^'^{xi,...,Xd^,yi,...,yd^,z) =< ^^'y(xi,...,a;do,So),*^\(?/i,...,ydi,Si) > . 

Applying this formula to the case Y = , we immediately obtain the following. 

Corollary 3.27. Let Xq, Xi be closed, oriented, non-spin, simply connected ^-manifolds with 
6"*" positive and odd. Let Pi be U{2)-bundles over Xi with W2{Pi) = W2{Xi), C2{Pi) = 1 
mod 2 and dim Mp. = 2di for some di > 0. For homology classes xi, . . . , Xdg, zq G H2{Xq; Z), 
yi, . . . , yd^, zi G H2{Xi; Z) with Xi ■ Xi = yj ■ yj = mod 2, zq ■ zq = Zi ■ Zi = 1 mod 2, we 
have 

^^^(a;i, . . . . . .,ydi,z) = <i/x,,,pX^i^ ■ ■ • ^^do) " ^Xi,Pi(z/i, • • ■,yd^) mod 2. 

Here z = Zq -\- Zi E H2{X; Z). 

Remark 3.28. In general, usual Donaldson invariants \E'p(xi, . . . ,Xd) do not lie in Z even if 
Xi G H2{X; Z). However if < W2{P),Xi >= mod 2 for all i then the value 'ifp{xi, . . . , Xd) 
lies in Z. Therefore the above formula makes sense. 
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As mentioned before, if there are no integers n such that dimM^^(^„) = then the proof 
of Theorem 13.261 is quite the same as that of Theorem 12.291 Hence we suppose Hypothesis 
13.191 To prove Theorem 13. 26^ we fix a sequence T" — )■ oo and consider manifolds X" = 
Xo U (F X [0, T"]) U Xi as usuaL As in Subsection 12. 3[ we can take sections : — )• A, 
s2 '■ Mx°' — C-Y. with the following property. Suppose that a sequence [A"] G Mx 

OL coil verges 

to . . . , m). Here [A^] e M^^(p(0)), [A^] G M^^^{p{i - l),p(0) = 1, . . . , r - 1), 

[A^] G Mj^^{p{r — 1)). Moreover suppose that p{i) are irreducible for i = 0, . . . , r — 1. Then 
we have 

4([A1)^sa([A-])K.--Ksa([A-]), 

sl{[A^])^sc{[A^])M-..msc{[Ar]). 

A standard dimension-counting argument shows that for any sequence [A"] G Mx-^iL^) = 
Mx^ n Va n Vc, there is a subsequence {[A°']}q,/ such that [A"'] converges to ([^[f], [^f]), 
where [A^] G M^,^(p;L^), [A^] G M^^{p-L^\Lp) for some < /3 < 3 and [p] G CF8„,,_^(y), 
or [A'^] G M^^j(6'„J, [y4~] G M^^(6'„J. Hence for large a we have 

Mx.(L3) = |J U M^^(p;L^)xM^^(p;L3\L^)U |J ^0(3)[^ n 14 n l^^. 

Here S'0(3)yj is the image of 5*0(3) by a gluing map as in the proof of Lemma [3. Ill Therefore 
we get 

P [p] \M 

Thus we must show 

1 if riQ = a + 1 mod 2, 



This follows from the following lemma. 

Lemma 3.29. For each [A] = ([Aq], [^i]), the restriction CY,\so{'i)^ is non-trivial. And the 
restriction A\so{3)iAi ^■^ non-trivial if and only if Uq = a -\- 1 mod 2. 

Proof. As in the proof of Lemma 13.141 we can see that a necessary and sufficient condition for 
the restrictions of the line bundles to be non-trivial is that the indexes Ind"*" Baq , Ind"*" '^Aq are 
odd. The calculations in the proof of Lemma 13.201 show that 

Ind"'' = zq ■ zq = 1 mod 2 

Ind"'' ;^Ao = no — a mod 2. 

Therefore we obtain the statements. □ 

4. A NON-EXISTENCE RESULT 

Let X be a closed, oriented, simply connected 4-manifold and Qx be the intersection form 
of X on H2{X; Z). For positive integers k and [Si], . . . , [T,2k] £ H2{X; Z), we define 

• • • ' P2A:]) := E Qx{[^a{l)], Pa(2)]) " " " ( [S^(2fc-1)] , Pa(2A:)])- 

■ <TG62fe 

In [1], Donaldson showed that when X is spin and = 1, Qx must satisfy the condition 
(32) gg)([Ei],...,p4]) = mod 2 
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for all [El],..., [E4] e H2{X;Z). 

In this section, we will consider the intersection forms of compact 4-manifolds with bound- 
aries homology 3-spheres. We prove that the condition also holds for compact, spin 4-manifold 
X with 6"*" = 1 and with boundary a homology 3-sphere Y, if HFi{Y; Z2) and HF2{Y; Z2) are 
trivial. Here HF^,{Y\ Z2) are the usual Floer homology groups with coefficients Z2. 

Theorem 4.1. Let Y he an oriented homology 3-sphere with HFi{Y; Z2) = 0, HF2{Y; Z2) = 0. 
IfY bounds a compact, spin, simply connected 4-'m(inifold X with b^{X) = 1, then we have 

gg)([Si],...,[S4]) = mod 2 
for any [Si], . . . , [S4] G if2(X; Z). 

This is originally due to Fukaya, Furuta and Ohta. (See [12] •) Making use of techniques 
developed in this paper, we write down the proof of this theorem. 

4.1. Relative invariants for spin 4-manifolds. In the previous section, we defined relative 
invariants for non-spin 4-manifolds with boundary. Here we define relative 2-torsion instanton 
invariants for spin 4-manifolds, which are used to prove Theorem 14. 1[ To define the invariants, 
we modify the construction in the previous section. 

Let Q = Y X SU{2) be the trivial S'f/(2)-bundle over Y and fix a G {0, 1}. For irreducible 
flat connections p, cr on Q with dim My (p, cr) < 2, we can take sections sa : MY^^{p,(y) 
A(°)(p, cr) which are compatible with gluing maps and satisfy the transversality conditions. 

Definition 4.2. Put Cj{Y) := CFj{Y) © CFj_i{Y). We define 9^'^) : Cj{Y) Cj^i{Y) as 
follows. Let /3i,/32 be integers with < f3i < (32 < I and take generators [p] G CFj_p^^, 
[a] G CF,.p,^^{Y). If /32 - /3i = 0, put 

< d^^\[p\), [a] >:= #M°,j,(p,a) mod 2. 

If /52 - /3i = 1, put 

< 9(")([P]), M >:= #(M°,j,(p,cr) n Va) mod 2. 

We define d^""^ by using these numbers as usual. 

Let {[A"]}^, be a sequence in My^]fj(p, a) for some fiat connections p, a with dim My-^^^p, cr) < 
2. We can show that if [v4°] converges to some ([A^], . . . , [A^]) G M^^^{p,p{l)) x ■■■ x 
My^]fj(p(r — 1), cr) then p{i) are irreducible for z = 1, . . . , r — 1. This means that we can prove 

as in Section [21 

Definition 4.3. &\y) := H,XC,{Y),d^°'^). 

Consider a compact, simply connected, spin 4-manifold X with boundary Y and h'^{X) = 1. 
We will define a relative invariant 

: i/2(X;Z)®^ — 

Take a S'f/(2)-bundle P over X and a trivialization ip of P|y with 

C2(P; ip)=2eH\X,Y;Z) = Z. 

We denote by 6 the trivial connection on P|y with respect to ip. The dimension of the moduli 
space Mj^{6) of instantons on X with limit 6 is 

8c2(P; - 3(1 + fe+(X)) = 8 ■ 2 - 3(1 + 1) = 10. 
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For generators [p] G CFp(Y), we have moduli spaces M-^[p) of dimension 10 — (3. 

Let E be a closed, oriented surface embedded in X. We put -Bs + = B^U {[^s]}- It is 
known that the determinant line bundle £s — >■ ^t, extends to ^. (See [8].) We use the same 
notation £s for the extension. We can take a section ss : -B^ _,_ — )■ jCs such that ss does not 
vanish near [6y\ ■ We always assume that sections of £s have this property. Let Vs be the zero 
locus of sy. and we define M-^{p) fl Vs to be 

{[A]GM^(p)|.s([A|d) = 0}. 

Take homology classes [Si], . . . , [S4] G H2{X;Z). For generic surfaces Si,...,S4, the 
intersection of any two surfaces is transverse and the intersection of any three surfaces is 
empty. Moreover for generic sections s^i, • • • , ss^, all generators [p] G CF^iY) and all subsets 
{li, . . . ,ls} C {1, . . . , 4}, the intersections 

s 

M^(p;S,„...,Sj = M^(p)n []Vj:^^ 

m=l 

are transverse. 

A standard dimension-counting argument shows the following: 

Lemma 4.4. Let p be aflat connection on Q. Assume that a sequence {[A°']}a in Mj^{p; Si, . . . , S4) 
converges to ii[A^],Z,),...,{[A^],Zr)). Here {[A^], Zo) G M^(p(0)) xSym"«(X) and{[Ar],Z,) G 
(MyxM(p(z - l),p{i)) X Sym"'(F x R))/M for z = 1, . . . ,r. 

(1) If [p] G CF2{Y) and dim M^(p) = 8 then r = 0, = 0. 

(2) If [p] G C-Fi(y) anc? dim M^(p) = 9 then r < 1 and all Zi are empty. Moreover when 
r = 1, p(0) is irreducible. 

(3) If p = 6, then one of the following holds: 

• r = 0, Zo = 0. 

• r = and Zq = {xi,X2} for some Xi G S^^ fl S^^, X2 G S^g H S^ with /i, . . . ,/4 

• r = 1, p(0) zs irreducible and all Z^ are empty. 

• r = 2, p(0),p(l) are irreducible and all Zi are empty. 

It follows from the lemma that we may take admissible sections sa(p) : Mj^{p; Si, ... , S4) — > 

A(p) for [p] G CFi(r). We define a chain ^/^^^ = ^/'^^(Si, . . . , S4) G Cf^(F) as follows. For 
[p] G CF2(r) put 

< i^x\ [p] >■= #(M^(p; Si, ... , S4) n Va) mod 2. 

For [p] G CFi(y), put 

< [p] >■■= #M^(P; Si, ... , S4) mod 2. 
These define a chain iIj"^ G c!f\Y) as usual. 

Lemma 4.5. T/ie c/iazn G C^°\y) is a cycle. Moreover the class \l/^^([Si], . . . , [S4]) = 

["^x] ^ -^2''''(^) independent of the metric and the sections and depends only on the homology 
classes [Si], . . . , [S4] G H2{X; Z). 



46 



HIROFUMI SASAHIRA 



4.2. Proof of Theorem 14.11 We start with the following lemma. 

Lemma 4.6. If both of HFj^i{Y;'Z2) O'nd HFj{Y \'L2) are trivial, then is also trivial. 

We can easily show this by considering the long exact sequence associated with a short exact 
sequence 

CF,_^{Y;Z2) ^ CF,{Y ■,Z2) — ^ 0. 

Next we define a map 

^(a) . j{a)^Y) ^ Z2. 

For [p] G CFi{Y), put 

D^^\[p]):=#M'y^^ip,9) mod 2 

and for [p] G CF2(r), put 

D^^\[p]):=if{M^,^ip,e)nVA) mod 2. 

Then we have a linear map D^*^^ from c!f\Y) to Z2. Counting the number of the ends of 
appropriate 1-dimensional moduli spaces, we get 

o d'^"^ = 0. 

Hence we obtain 

Jj(a) . j{a)^Y) ^ Z2. 

(We denote the map by the same notation.) 

Theorem 14.11 immediately follows from Lemma 14.61 and the following proposition. 

Proposition 4.7. The image of • • • . PJ) ^ /^^(i^) by L>(°) zs 

q2)([Ei],...,[S4]) mod 2. 

To prove this, we consider the ends of the cut-down moduli space M-^{9; Si, ... , S4) fl Va 
with dimension 1. Let {[A"]}^ be a sequence in the cut-down moduli space. Then there is 
a subsequence {[A"' ]}a' which goes to (([^0]) Zq), . . . , ([^4^], Z,,.)). We can see that we are in 
one of the following cases: 

(1) r = 0, Zo = 0. 

(2) r = 0, Zq = {xi, X2} for some Xi E S;^ fl S/2, X2 E S;,, fl with /i, . . . , ^4 distinct and 
[A^] = [Ox]- 

(3) r = 1, all Zi are empty, and [p(0)] G CF2{Y), [A^] G M^(p(0); Si, . . . , S4), [A^] G 
M°,Jp(0),^)nV^A. 

(4) r = 1, [p(0)] G CFi(r), G M^(p(0); Si, . . . , S4) n Va, [A^] G , Jp(0), ^^). 
We describe the ends of the moduli space relevant to the case ([2]). Let Xi G S;^ fl S^^, 

X2 G S/g n S^ for distinct /i, . . . , /4 and take small neighborhoods f/j;^, in X. There is an 
S'0(3)-equivariant map 

k:U^^x X 50(3) X S0(3) x (Tq, 00) x (Tq, 00) — > /f^^ = H+{X)®5u{2) 

such that K~^(0)/5'O(3) is homeomorphic to an open set in M-^{9). As in [1] we can see that 

(/s:-i(0)/5O(3)) n Fs, n ■ ■ • n = {a^J X {0:2} X £ X {1} X { (T, T) I T > To }, 
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where £ is a loop in S0{3) which represents the generator of tti (5*0(3)) = Z2. For each 
x = (xi, X2), we have a gluing map 

Gh:ix{ {T,T) \T>To } ^ M^(e; Si, . . . , S4) 

which is given by gluing the trivial connection on X and two copies of the fundamental in- 
stanton J over at xi,X2- The intersection of the image of this map and Va is one of the 
ends of the moduli space. 

Let [A] = {[Ao],[Ai]) be an element of M;^ (p; Si, . . . , S4) x {M^^^{p,e) n Va) for [p] e 
CF2{Y), or an element of (M^(p; Si, . . . , S4) n Va) x M^^^{p, 6) for [p] G CFi{Y). Then we 
have a gluing map 

Gl^A] ■■ (To,oo) ^M^(^;Si,...,S4). 
The ends of the moduli space relevant to the case ([3]), (jl]) are the intersections of the images 
of these gluing maps and Va. 
Put 

M' = M^(^;Si,...,S4) \ (|JlmG/^u|JlmG/[^). 

X [A] 

If we perturb the section sa over a compact set in M' then the intersection M' fl Va is a smooth 
compact manifold with boundary 

U M^(p; Si, ... , S4) X (MO ,^(p, 6) n Va) 
[p] 

U U (M^(a; Si, . . . , S4) n Va) x M^,^{cx, 6) 

u[jGi^{e,To)nVA. 

X 

Here [p] runs over the generators of CF2(Y) and [a] runs over the generators of CFi{Y). 
Therefore we get 

D(°)(^:^i([Si], . . . , [S4])) + J2 ^GUi, To) n V^a) = mod 2. 

X 

We can see that 

4^{GUi,To)nVA) = l mod 2 
for each x. This follows from arguments similar to those which deduce fl3T|) . The point is that 
J is an instanton on an S'f/(2)-bundle P' with C2(P') = 1 and hence the index IndDj is odd. 
Therefore we get 

dW(M/-([Si], . . . , [S4])) = 5^ 1 = Q^x\m P4]) mod 2 

X 

as required. 
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^ ■ 

o ■ 
(N : 

Abstract. We construct a variant of Floer homology groups and prove a gluing formula 

^ , for a variant of Donaldson invariants. As a corollary, the variant of Donaldson invariants is 

' non-trivial for connected sums of 4-manifolds which satisfy a condition for Donaldson invari- 

^^.^ I ants. We also show a non-existence result of compact, spin 4-manifolds with boundary some 

' homology 3-spheres and with certain intersection forms. 

1. Introduction 

In this paper, we construct a variant of instanton Floer homology groups and prove a gluing 
^ . formula for a variant of Donaldson invariants introduced in [19] . 

! As S. K. Donaldson showed in [H El EJ [7], the moduli spaces of instantons have crucial 
^ ' information about the topology of closed, oriented, smooth 4-manifolds X. Mainly there are 
two methods to draw out the information from the moduli spaces. First one is to describe the 
psj structures of the singular points and the ends of the moduli spaces [HE]. The description gives 
^ , strong restrictions to the realization of unimodular quadratic forms as the intersection form of 
^ [ a smooth 4- manifold. Second one is to integrate cohomology classes over moduli spaces P [7]. 
^ ■ This gives differential-topological invariants of 4-manifolds which distinguish different smooth 
^ . structures on the same topological 4-manifold. 

^ I It used to be hard to compute Donaldson invariants in general. However instanton Floer 
(yQ I theory gives us a way to compute Donaldson invariants "^x when X has a decomposition 
O • X = Xq Uy Xi for some compact 4-manifolds Xq and Xi with boundary Y and Y. Here Y is 
>■ . a closed, oriented 3-manifold and Y is Y with the opposite orientation. A. Floer introduced 
^ I instanton homology groups HF^,{Y) for homology 3-spheres Y in [TT]. Floer 's groups allow us 
^ ■ to generalize Donaldson invariants for compact, oriented 4-manifolds Xq whose boundaries are 
homology 3-spheres Y . The relative invariant ^ Xo is an element of HF^{Y), which is defined 
by integrating cohomology classes of the moduli spaces of instantons over Xq = XqU{Y x M>o). 
Assume that a closed 4-manifold X has a decomposition X = Xq Uy Xi for some homology 3- 
sphere Y and two compact 4-manifolds Xq, Xi with boundary Y, Y and with 6+(Ao), b~^{Xi) > 
1. Here b'^ stands for the dimension of a maximal positive subspace of the intersection form 
on H2{X). Then we have the relative invariants \I'xo ^ HF^iY), Xi ^ HF^iY). There is a 
natural pairing 

<, >:HF,{Y)®HF,{Y)^Q, 

and we have a relation 

(1) ^x =<^x,,^x,> ■ 

Instanton Floer homology groups also enable us to generalize the results on the non-existence 
of closed 4-manifolds with fe"*" = and with non-standard intersection forms. Donaldson |8] 
showed that if HFiiY) = then there is a restriction to the realization of unimodular quadratic 
forms as the intersection form of a 4-manifold Xq with boundary Y and with = 0. The 
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proof involves the description of the singular points and the ends of the moduli spaces of 
instantons over Xq as in the case of closed 4-manifolds. We also refer to K. A. Froyshov's 
papers [121 EH US]- He gave obstructions for (rational) homology 3-spheres to bound a 4- 
manifold, using Seiberg-Witten theory and "h invariant" defined by instanton Floer theory. 

On the other hand, there are some variants of Donaldson invariants. R. Fintushel and R. 
Stern defined a variant of Donaldson invariants for closed, simply connected, spin 4-manifolds in 
|1U] . They used a 2-torsion cohomology class Ui G H^{Mp; Z2) of moduli spaces Mp which was 
originally defined by Donaldson in [5J. In S. Akbulut, T. Mrowka and Y. Ruan extended the 
construction of Ui to non-spin 4-manifolds and they showed a universal constraint of Donaldson 
invariants for non-spin 4-manifolds. Using ui for non-spin 4-manifolds, the author [19] defined 
a variant of Donaldson invariants for non-spin 4-manifolds, which can be regarded as an 
extension of Fintushel-Stern's invariant to non-spin 4-manifolds. A remarkable feature of these 
variants is that they are non-trivial for connected sums of the form X = Y^S"^ x S"^ in general. 

In [17], it was announced (without proof) that K. Fukaya, M. Furuta and H. Ohta showed 
a non-existence result of compact, spin 4-manifolds with 6"*" = 1 and with some intersection 
form when their boundaries are diffeomorphic to some homology 3-spheres. In [T7], Furuta 
explained that a variant of instanton Floer homology groups and an extension of to 
compact, spin 4-manifolds with boundary a homology 3-sphere are used in the proof. This is 
a nice generalization of a result in [5], however, explicit examples were not given. 

The main purpose of this paper is to construct a variant of instanton Floer homology groups 
and to extend the relation ([1]) to \E'^\ As a corollary, we obtain a non- vanishing result for 
connected sums of some 4-manifolds. Moreover we will write down a proof of Fukaya-Furuta- 
Ohta's non-existence result using the variant of instanton Floer homology groups. In the 
proof, we make use of the structure of the ends of some moduli spaces as in [5]. There is a 
difference between our variant of instanton Floer homology and Fukaya-Furuta-Ohta's one. In 
our construction we use the determinant line bundles of families of 9-operators over 7 x M. 
Here 7 is a loop in Y. This enables our gluing formula to be applied to not only the case when 
4-manifolds are spin but also the case when 4-manifold are non-spin. 

In [12], the author showed that the variant of Donaldson invariants is non-trivial for 2CP^#C^. 
As we will explain in Section [31 this calculation is suggesting that we need to take into account 
the following aspects in the construction of our variant of instanton Floer homology groups: 

• The trivial fiat connection on Y should play an important role in the gluing formula 
for ^^1. 

• We need a similar construction to Fukaya's extension [16j of instanton Floer homology 
groups. 

In the gluing formula for the usual Donaldson invariants, the trivial fiat connection has no 
contribution. Hence the contribution of the trivial fiat connection to the gluing formula for 
is a new phenomenon. See Donaldson's book [8] and Froyshov's papers [lU |15] for other 
treatments of the trivial fiat connection. In Fukaya's construction, we use the determinant 
line bundles of 9-operators over 7 x M, and we need sections of the determinant line bundles 
with certain properties. The properties have to do with non-compactness of the moduli spaces 
over y X M and transversality. (See Proposition 12.81 ) We call sections having the properties 
admissible sections. One of the main purposes of this paper is to give a construction of 
admissible sections of determinant line bundles and it will be done in Section [2J 
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Instanton Floer homology groups are defined for any liomology 3-spfieres. For otlier 3- 
manifolds Y, there are some problems coming from reducible fiat connections over 3- manifolds 
and the decomposition of homology classes of X along Y. In order to overcome the first 
difficulty, D. M. Austin and P. J. Braam [2] introduced an equivariant version of instanton 
Floer homology for 3-manifolds Y with bi{Y) = or i7i(y;Z) torsion free. Kronheimer and 
Mrowka [18] gave another method to treat reducible solutions in the context of Seiberg-Witten 
theory. On the other hand, Fukaya's construction [16j gives an effective way to overcome the 
second difficulty when there are no reducible fiat connections. However it does not seem that 
we have a complete method to construct suitable instanton Floer homology groups fitting to 
the situation where we treat both the problems at the same time. To construct a gluing formula 
for our invariant we must treat both the problems at the same time, even when the 3- 
manifold F is a homology 3-sphere. Although we treat only homology 3-spheres in this paper, 
we can regard our construction as a first step to give a method simultaneously applicable to 
both the problems. 

The organization of this paper is as follows. In Section |21 we recall Fukaya's construction. 
We will basically follow [16] and j3], however, with some modification. In Section |3l making use 
of techniques developed in Section [21 we will introduce a variant li"'\Y;'~f) of instanton Floer 
homology groups for homology 3-spheres Y, loops 7 in F and a G {0, 1}. We define relative 
invariants "^^^ G li"'\Y;'~f) for compact 4-manifolds Xq with boundary Y. We also prove a 
formula for \E'^^ similar to ([T]) (Theorem 13. 26p . In particular, we deduce a non- vanishing result 
for connected sums of 4-manifolds (Corollary I3.27p . In Section HJ we will prove the result on 
the non-existence of spin 4-manifolds with boundaries some homology 3-spheres and with some 
intersection forms (Theorem 14. ip . 

Acknowledgments . The author is grateful to Mikio Furuta for his suggestions. He would 
also like to thank Yukio Kametani and Nobuhiro Nakamura for useful conversations. 

2. FUKAYA-FLOER HOMOLOGY 

Let X be a closed, oriented 4-manifold with 6+ > 1 and take a f/(2)-bundle P over X 
and assume that the dimension of the moduli space of instantons on P is 2d for some integer 
d >0. We can associate cohomology classes /i([S]) of degree 2 on the moduli space to homology 
classes [S] G H2{X]Z). Here S is a closed, oriented surface embedded in X which represent the 
homology class. The cohomology classes /i([S]) are the first Chern classes of the determinant 
line bundles of a family of twisted d operators on E. Roughly speaking Donaldson invariants 
are the evaluations of cup products /i([Si]) U ■ ■ ■ U /i([Sj) on the moduh spaces. 

Suppose that we have a decomposition X = Xq Uy Xi and that the surfaces are split 
into two surfaces with boundary 7^ = along Y. We consider how to recover the Donaldson 
invariants of X from relative invariants of Xq and Xi. Fukaya's construction [16] provide 
us suitable homology groups to define the relative invariants. In the construction, we need 
sections of determinant line bundles of families of twisted d operators on 7/ x R which behave 
nicely on the ends of moduli spaces of instantons over F x R and satisfy certain transversality 
conditions. We call such sections admissible. The main point of this section is the construction 
of admissible sections of the determinant line bundles (Proposition 12. 8p . 

2.1. Outline of proof of gluing formula. In this subsection, we give a sketch of the con- 
struction of a gluing formula for Donaldson invariants. 
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Let X be a closed, oriented, simply connected smooth 4-manifolds with b'^ > 1 and odd, and 
P be a t/(2)-bundle over X with W2{P) non-trivial. We write Pdet for the f/(l)-bundle over X 
induced by P, and fix a connection adet on Pdet- We take a Riemannian metric on (7 on X and 
suppose that the virtual dimension of Mp is 2c? with d > 0. Here Mp = Mp{g) is the moduli 
space of instantons on P which induce the connection a^et on Pdet- The moduli space Mp is a 
smooth manifold for generic metrics g, and a choice of an orientation of the space H^{X) of 
self-dual harmonic 2-forms on X orients Mp. The moduli space Mp gives us the Donaldson 
invariant 

^x,p:H2{X;Zf''^Q. 

This is defined as follows. 

Let [S] be a class in H2{X; Z). We denote by the space of gauge equivalence classes of 
framed, irreducible connections on the restriction P|e of P to S which are compatible with 
Odetls- We have the determinant line bundle 

£s = detlnd{(9;^}[^]gg^ ^ i3s 

of the family of operators {^A}[yi]eBE' Here 

is the (9 operator twisted by the rank-two complex vector bundle E associated to P and a 

square root of the canonical line bundle of S, and 9^ is the adjoint. 

There is a natural action of SU{2) on B^, and the action of {±1} C SU{2) is trivial. Hence 
we have the SO (3) = SU(2)/ ± 1 action on B^. The quotient space B^/SO{?>) is the space 
B^ of irreducible connections on P|s which are compatible with Odetls- Since the action of 
{±1} C SU{2) on is trivial, we have the line bundle 

Cf := £f/50(3) ^ . 

Note that £f ^ may not be a square of a genuine line bundle. Take a section ss of £f ^ and 
denote the zero locus by Vj^. We define Mp fl l^s by 

Mpnv^ = { [A] e Mp I [A\j,] G l^s }- 

Since the restriction of elements of Mp to S may not be irreducible, this is not well-defined. 
But we can avoid this problem by replacing E with a small neighborhood of S. (See [H].) 
Take d homology classes [Si], ... , [S^] G H2{X] Z). We can show that the intersection 

Mp n n - - - n Fs, 

is transverse and finite for generic surfaces Si, . . . , and sections s^i, . . . , s^^- Since Mp 
and are oriented, we can associate each point of the intersection with a sign. We define 
\l/x,p([Si]5 - - - 5 Pcz]) to be 1/2'^ times the number of points of the intersection counted with 
sign, i.e. 

vi/^,p([Si], . . . , [s,]) = ^#(Mp n n - - - n K^J. 

We can see that this is independent of the choices of metric and sections. 

Suppose that X has a decomposition X = Xq Uy Xi. Choose a Riemannian metric qy on 
Y and let go,gi be Riemannian metrics on Xo,Xi whose restrictions to Y are equal to gy- 
Assume that the dimension of the moduli space Mp is 2d with < < 3. Take homology 
classes [Ei], . . . , [E^^] of X represented by surfaces Ei,...,E(i which intersect with Y. Put 
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= n Xo, = n Xi and 7^ = T,i HY. We assume that 7^ are diffeomorphic to 
for all /. We show how to compute . . . , [^d]) from data of Xo,Xi briefly under this 

situation. We use some facts about instantons which can be found in [8J. 

Take a sequence {T"}^^ of positive real numbers which diverges to infinity. We have 
manifolds X" = Xq U (y x [— T",T"]) U Xi which are diffeomorphic to X. The Riemannian 
metrics Qy, go, Qi induce Riemannian metrics (7° on Take instantons [^4°] G Mp[g°'). 

Then there is a subsequence { [A" ] }q,/ such that 

Here [A^] e M^^(p(0)), [A^] G Mj^^{p{r - 1)) are instantons over Xo = XqUY x M>o, Xi = 
xM>o which converge to projectively flat connections p(0), p(r — 1) at infinity. M-^ (p(0)) 
is a moduli space of instantons over Xq with limit p(0) and similarly for Mj^^{p(r — 1)). For 
i = 1, ... ,r — 1, [Af] G My-^]r(p(z — l),p(z)) = MyxiR(p(« — l),p{i))/M. are instantons over 
F X M with limits p{i — 1), p{i). ( The action of R on MyxR(p(« — l),p(i)) is defined by 
translations. ) Since dim Mp is less than 8, bubbling phenomena do not occur. As we will see 
below, we can take sections sf of — > Mpi^g") such that 

(2) sli[A"]) s^,(m)MsrMr])^---^srMr-i])^s^J[Ar]), 

I I 

where = U (7/ x M>o), = U (7; x M>o), = 7^ x M, and s^, s^^xM., sg;; are sections 

of line bundles defined by families of twisted d operators on the surfaces. (See Definition 12.71 
and Proposition 12. 8[ ) Suppose that all [A°'] lie in the intersection 

then at least one of the components of the limit of vanishes. A dimension counting 

argument shows that r is 1 and that 

m e M^^(p(0);L) = M^^(p(0)) n fl Vg,, [A^] G M^^(p(0); L'^) = M^^(p(0)) H f| V^,. 

Here L is a subset of {1, . . . ,d} and L'^ is its complement. We denote the number of elements 
of L by I L I . Then we also have 

(3) dimM^^(p(0)) = 2|L|, dimM^^(p(0)) = 2|L'=| 

and M^^(p(0); L), M^^(p(0); L'^) are finite (if the sections over the moduli spaces satisfy some 
transversality conditions and behave nicely on the end of the moduli spaces). A standard 
theory of gluing of instantons shows that 

Mp(^7") n n ■ ■ ■ n Vi, - U U (p; l) x m^^ (p; l'^). 

L [p] 

Here [p] runs over the gauge equivalence classes of fiat connections satisfying ([2]). This implies 
that 

(4) ^x,p([Si], . . . , [S,]) = J2Y1 #^Xo(P; L) ■ #M^^(p; L^). 

L [p] 

From this formula, formal sums 

L [p] i [p] 
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recover the Donaldson invariant. Here 

n,{p-L) :=i^Mj,^ip;L), n,{p-L^) := #M^^ (p; L'^). 

We consider tpx^ as an element of the vector space CFF{Y] 7) spanned by a set 

{ [p] ® 7l I and p satisfy ^ }. 

The formal sums ipxo-, '4'Xx depend on the metrics and sections. We will define a boundary 
map 

d : CFF{Y-f) — > CFF{Y-f) 

such that the composition d o d is identically zero, and show that diljxo = and the class 
= bPxo] £ HFF{Y; 7) = H{CFF{Y; 7), d) is independent of the metric and sections. 
There is a pairing 

< , >: CFF(Y; 7) ® CFF{Y; 7) ^ Q 

such that [p] ® 7l and [p] ® 7l<= are dual to each other. We can see that the pairing induces a 
pairing 

HFFiY; 7) ® HFFiY; 7) — ^ Q. 

The formula (jl]) implies 

^x,p([Si],...,[S,]) =< ^Xo,^x, >. 

This is the gluing formula. 

2.2. Fukaya's construction. Let F be a closed, oriented 3-manifold. Take a Riemannian 
metric on Y and a ?7(2)-bundle Q over F, and fix a connection Odet on the [/(l)-bundle 
Qdet induced by We consider connections on Q which induce the fixed connection a^et on 
Qdet- Let Aq be the space of connections on Q with fixed determinant and Qq be the space of 
automorphisms of Q of determinant 1. The Chern-Simons functional is an 5*^- valued functional 
on the quotient space Bq = Aq/Qq. The critical points are the gauge equivalence classes of 
projectively fiat connections. If the Hessian of the Chern-Simons functional at a projectively 
fiat connection is non-generate, we say that the projectively fiat connection is non-degenerate. 
For simplicity, we will refer projectively fiat connections over Y with fixed determinant as "flat 
connections" . Throughout this section we assume the following hypothesis. 

Hypothesis 2.1. All fiat connections on Q are irreducible and non-degenerate. 

We are always able to perturb the Chern-Simons functional such that any critical points are 
non-degenerate. See [11], [8]. 

Let R{Y) = R(Y, Q) be the set of gauge equivalence classes of flat connections on Q. It 
follows from Hypothesis 12.11 that R{Y) is a finite set. We define a Zg-grading function 5y on 
R{Y) as follows. Let tt be the projection from y x M to y. Fix a flat connection po on Q. 
For each flat connection p on Q, choose a connection Aq = Aq{p,pq) over y x M, which is 
compatible with 7r*adet; such that 



Ao 



TT*p on F X (—00, — 1), 
7r*po on F X (1, 00). 



Then we have an operator 

(5) Da, = dX + 4o • ^'(AyxR ® ^*0q) ^'((AyxR © A^xm) ® ^*Qq)- 
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Here gq is the bundle of trace free, skew adjoint, endomorphisms of the rank-two complex 
vector bundle E associated with Q. Under Hypothesis 12 .11 this operator is a Fredholm operator 
and we have the numerical index ind Dj^^ G Z. We set 

5y([p]) = indDAo mod 8. 

We can show that this depends only on the gauge equivalence class of p (and po)- 
For j G Z we write CFj{Y) for the Q- vector space spanned by 

{[p]eR{Y)\5Ym^j mods}. 

Let d be an integer with 1 < d < 3 and 7; = S*^ be a loop in F for / = 1, . . . , c?. We write 7 for 
{li}i=i- We define the Fukaya-Floer chain group CFF^:{Y;'y) by 

d 

CFF,(y;7):=0 CF,-2/3(r) ® Q < 7l >, 

/3=0 Lc{l,...,d} 

|L|=/3 

where jl '■= Ih ■ ■ ■ lip ^ Sym Q < 71, . . . , 7^ > for L = {/i, . . . , Ip). We define a boundary 
operator 

d : CFF,{Y-^) CFF,,,{Y;j) 

as follows. Take two generators 

[p] ®7L, e CF^^2^,{Y)®Q< 7L, >C CFF,{Y;j), 

[a] ® 7L2 e C7F,_2&-i ®Q<lL, >C CFF,-_i(F;7). 

Then we have a moduli space MyxR(p, o") of instantons with limits p, a and the dimension is 
2(/32 — + 1- We write My^^ip, ex) for the quotient Myxr{p, <^)/^, where the action of M is 
defined by translations. When Li G L2, we define 

(6) < d{[p] ® 7lJ, M ® 7L. >:= " < ci(£f ) U ■ ■ ■ U Ci(£f/ ), [M° , Jp, a)] > " , 

where {/i, . . . , Z/?} = L2\Li and are line bundles defined by families of twisted d operators 
over 7;. x M. Since the moduli spaces are non-compact in general, we must specify the meaning 
of the pairing, and it will be done later. When Li is not included in L2, we define < d{[p] ® 
M ®7l2 > to be zero. The matrix elements < d{[p] ®7li), [o"] ®7l2 > give the boundary 
operator d. We will show that d o d is identically zero. The Fukaya-Floer homology group 
HFF^,{Y; 7) is defined to be the homology group of {CFF^:(Y; 7), 9). 

We give the precise definition of ([6]). To do this, we introduce some spaces of connections 
on y X M and 7/ x M. For fiat connections p, a on Q, take a smooth connection Ao = Aq{p, a) 
on 7r*Q as before. Let r > be a small positive number and we set 

^YxrIp. (t) := { Ao + a \ a e ll'^iAy^^ O 'k*Qq) }. 

The wighted Sobolev space L4 ^ is defined as follows. Take a function Wr on F x M such that 

Wr{y, t) > for ^{y, t) eY xR, 
iy^(l/,t) = e^l*l for |t| > 1. 

For a smooth, compact supported section /, we define the weighted L|-norm by 

k=0 



8 



HIROFUMI SASAHIRA 



The weighted Sobolev space L^''^ is the completion of the space of smooth, compact supported 
sections. Note that since p, a are irreducible, every connection in Ay^RiP: f^) is irreducible. 
By Hypothesis 12. ![ we need not to introduce the weighted Sobolev space for the construction 
of the moduli space. However we need the weighted Sobolev space to define the determinant 
line bundles as explained below. 

We introduce a gauge group acting on the space of connections. We put 

Gyxr ■= { 9 ^ Ll i^^{Antn*Q) \ dA^g ■ g'^ e }■ 

Then Qyxr acts on AyxR^Pi by the gauge transformations. We denote the quotient Ayy,^/ Qyxr 
by ;By^]g(p, (t) . For g G QyxRi we have g{y,t) — )■ 1 or g{y,t) —1 as t ±oo. ( See Propo- 
sition 4.7 in [8j. ) We set 

^YxR ■= { g ^ Syxr I lim g{y,t) = 1 }. 

t— s>±oo 

We write Byy^^^ip, cr) for Ayxr{Pj ^)/^yxR- Since Qyxr/QyxR is isomorphic to Z2 x Z2, we have 
a natural action of Z2 XZ2 on By^^^ip, cr), and By^^^p, a) is identified with By^^{p, o")/Z2 XZ2. 

Let MyxM(p, cr) C By^^{p, a) be the moduli space of instantons with limits p, a. We can per- 
turb the instanton equation such that the operators Da are surjective for all \A\ E My^^{p, a) 
( [m E] ) • For simplicity, we always assume the following. 

Hypothesis 2.2. The operators Da defined by (jS]) are surjective for all [A] E My^^^iPjCr). 

Under this hypothesis, the moduli space MyxiR(P)'^) is smooth of expected dimension. 
Next we introduce spaces of connections and gauge groups over = 7; xM. Let Ai = Ai{p, a) 
be the restriction of the fixed connection Ao{p, a) to F^. We set 

Ar,{p,(T) :={Ai + a\aE L^'"(A^^ ® 7r*0Q|rJ }, 

S'r, --={9^ (Autvr^girJ | dA,g ■ g'' e LI'\ lim g = l}. 

' ' t— >-±oo 

We denote the quotient space Ari{p,cr)/Q^^ by Bri{p,cr). Note that the restrictions of p,a to 
7; may be reducible, and hence some connections in Ari{p,cr) are reducible. 

We define the determinant hne bundle over -Br, (p, cr). We need a spin structure on F;. Since 
i7^(F;; Z2) is isomorphic to Z2, there are two spin structures on each F; (up to isomorphism). 
We fix a spin structure on 7; which represent the trivial class in the 1-dimensional spin bordism 
group. This spin structure induces a spin structure on F/. We use this spin structure. (We will 

explain the reason why we take this spin structure in Remark 12.31 below.) The spin structure 

1 

induces a square root K^^ of the canonical line bundle i^'r, • For connections A E Ari (p, cr) we 
have the twisted d operators 

Ba : lI'^'^'^\kI ® 7r*E|rJ ^(-"'"^(A^f ® n*E\r,). 

Here L^'^''^''^^ are the weighted Sobolev spaces with weight function VF^ > such that 

W^{y,t) = e^' for |t| > 1. 

We do not take the absolute value of t in the exponent this time. The operators are Fredholm 
operators for small r > 0. Since we have the universal bundles 

Er, := Ar,ip,a) Xgo^ (7r*E|rJ — > Br, x F,, 
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we obtain complex line bundles 

Ci{p,cr) = (detInd{(9A}[A])* — > -Br;(p,cr). 

Let fi be the map from Byxr{P: o") to i3r,(p, cr) defined by restricting connections to Ti. Then 
we have a natural action of Z2 x Z2 on the pull-back flCf^ and the action of the diagonal 
Z2 = {±(1, 1)} is trivial. Hence we get the line bundle 

Cf\p,a) := r7£f2(p,a)/Z2 x ^ B*y^^{p,a) 

for each I. 

Remark 2.3. We explain the reason why we choose the spin structure on Ti induced by 
the spin structure on 7^ representing the trivial class of 1-dimensional spin bordism group 
^spin _ When we prove the gluing formula for Donaldson invariants, we consider Ti as a 
neck of a closed surface in a closed 4-manifold. We need the restriction of a spin structure on 
the closed surface, which is used to define Donaldson invariants, to F;. This spin structure is 
induced by the spin structure on 7; which represent the trivial element in fij^*"". 

To define ([6]) we need sections si = si{p, a) of Cf^i^p, cr) which behave nicely on the ends of 
the moduli spaces and satisfy suitable transversality conditions. We briefiy recall some basic 
definitions and facts which are relevant to the end of the moduli spaces. (See [B] for details.) 

For a real number T let ct be the translation 

r X M — > r X M 

{y,t) ^ {y,t + T). 
We call a sequence T of real numbers 

Ti < ■ ■ ■ < T,_i 

a translation vector. 

Definition 2.4. Let be a sequence in MyxR(p, cr). We say that {[A"']}^ is weakly 

convergent to 

{{[Ar],Z,),...,--- ,{[Ar],Zr)) 

for some ([A], Z,,) G {Myxmipii - 1), p(0) x Sym"'(y x M)) /M if there is a sequence {T"}„ of 
translation vectors with 

as — i- 00 such that for each i the translates c^ti([A°]) converge to [A°°] over any compact 
sets of {Y X ]R)\Zi and \c^a{FAc)\'^dpYxR weakly converge to \FA°°\'^dpYxR + J2?=i^zr Here 
dpYxR is the volume form on Y x R, Zi = [zi, . . . , z^J and 5^, are the delta functions. 

Proposition 2.5. Any sequence in Myy^j^ipjCr) has a weakly convergent subsequence. 

Let {[A"]}q, be a sequence of My^^^ip, cr) which weakly converges to (([A^], Zi), . . . , {[A"^], Zr)). 
It follows from the additivity of the index of the operator Da that if the dimension of 
MY^-^{p,a) is less than 8 then Zi are empty for all i. In Section [2] and [3l we only con- 
sider the case when dim My^^^ip, cr) < 8 and hence Zi are always empty. In Section HI we will 
analyze the end of moduli spaces under the situation where Zi are not empty. 

The end of the moduli spaces are described by gluing maps. We consider the case when Zi 
are empty. Let Ui, . . . ,Ur be precompact, open sets of My^^{p, p(l)), . . . , My^^^p^r — 1), cr). 
Then we have a gluing map 

Gl:U,x (To, 00) X ■ ■ ■ X (To, 00) x t/, ^ M^^^{p, a) 
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for some To > 0. The map Gl is a diffeomorphism onto its image. 

Proposition 2.6. Let {[A°']}a be a sequence in My^jg(p, a) converging to some {[Af^], . . . , [A'^]) G 
Ui X ■ ■ ■ X Ur- Then for large a, [A°'] are in the image of the gluing map. 

We also have a gluing map 

Gl : Cf\p, pil))\u, Cf{p{r - l),a)\u^ ^ £^\p,a)\,^Gi ^ 

which covers Gl. For T = (Ti, . . . , T^-i) with Tj > Tq, we write GIt_ for the restriction of Gl 

to Ui X {Ti} X ■■■ X {Tr-i} X Ur. 

Using these definitions and facts, we state the properties of sections of the line bundles which 
are required to define (Q. Let {[A"]}^ be a sequence in Myy^^ip, <j) with limit • • • , [^J^])- 

By the above proposition, for large a, there are instantons [Af] G Ui and T° > Tq such that 

[A"] = G/([A?],Tf,...,T,%,[AJf]). 

Definition 2.7. Let si be sections of the line bundles Cf'^{p,a). Under the above situation, 
we say that converge to si{[A'^]) K ■ ■ ■ K si{[A^]) if 

WsiiiA'^]) - gIto' (sKKl) K • • ■ K sKK])) II 

as a — 7- oo. Here || ■ || is the norm on Cf'^{p, a) induced by the L^''^~^''^^-norms on the spaces of 

sections of K^^ ® tt*E, Ap'' O K^^ O tt*E. 

The following proposition is the key in this paper. 

Proposition 2.8. For flat connections p,a on Q with dim My ^]g(p, a) < 8, we have sections 
Si{p,a) : My-^-^ip, (t) — )■ Cf'^{p.,a) which have the following properties: 

(a) For any sequence {[A"]}a in Myy^^XPi ^) converging to some . . . , [^5^]), 

si{\A-])-^si{[Ar])M...msi{m) 

in the sense of Definition \2. 7| 

(h) Let Vi = Vi{p, a) be the zero locus of Si{p, a). For L C {1, . . . ,d} with dim My^^^ip, cr) < 
2|L|, the intersection 

M° ,jj(p, a; L) := M^^^ip, a) n f| 

is empty. 

(c) If dim My ^{p, a) = 2\L\, the intersection MY^^{p,a; L) is transverse and compact. Hence 
the intersection is a finite set. 

Here we introduce the following definition. 

Definition 2.9. If sections si{p, a) of Cf^^p, a) have the properties in Proposition 12.81 we call 
them admissible. 

The proof of Proposition [23] will be given in the following two subsections. In this subsection, 
we assume that we have admissible sections si{p, a) and define the boundary operator for the 
Fukaya-Floer homology groups. 

Let d be an integer with 1 < d < 3. For subsets Li C L2 C {1, . . . ,d} and flat connections 
p,a with dim My (p, cr) = 2|L2\ivi|, the intersections Myxjg(p, a; L2\Li) are finite by the 
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property (|c]) in Proposition 12.81 Hence we can count the number of points in the intersections. 
They numbers give the definition of (E]). More precisely we need to attach a sign ±1 to 
each point. However we mention nothing about signs. (The main purpose of this paper is 
to construct variants of Floer homology groups for 2-torsion instanton invariants. They are 
defined over Z2 and we do not need signs for the construction. ) 

Definition 2.10. Let 7 = {7«}f=i be a set of loops in Y, where d is an integer with 1 < d < 3. 
For integers /3i,/32 with < l3i < < d, take generators [p] G CFj_2/3i(F), [a] E (7Fj_2/32-i(^) 
and choose subsets Li, L2 of {1, ... , d} with = f3i, IL2I = (32- Then we put 

We define d : CFFjiY;^) -> CFFj^iiY;^) by 

d{[p] ® 7li) 5^ 5^ 5^ < ^([P] ® 7lJ, M ® 7l. > M ® 7l.. 

/32 [<t] i2 

We prove the following. 

Lemma 2.11. d o d = 0. 

This is given by counting the number of the ends of 1-dimensional moduli spaces. Let Pi, P2 
be integers with Q < Pi < P2 ^ d. Choose generators [p] G CFj_2Pi{X)^ [o] G CFj_2/32-2(^) 
and Li C L2 C with = IL2I = ^2- We have a reduced moduli space 

My^jg(p, cr) of dimension 2(/32 — + 1 (< 8). Suppose that we have a sequence {[A"]}^ in the 
intersection My^]g(p, a; L2\Li) of formal dimension 1 which converges to some ([^^], • • • , [A^]) 
with r > 1. (Note that Proposition 12.81 does not assure that My ^^(p, cr; L2\-Li) is transverse, 
since dim My x]jj(p, cr) > 2\L2\Li\.) First we show that r = 2. Put 

m = { / G L2\Li I .K[Ar]) = }. 

For I G L2\Li and all a, si{[A'^]) = 0, and siHA"]) converges to S;([A^]) K • • ■ K ^^([A^]). 
Hence there is a number G {1, . . . , r} such that 

for each / G L2\Li. This means that / G L2\Li lies in L{i{l)). Therefore we have 

r 

P2-Pi = \L2\L1\ <Y,\Hi)\- 

i=l 

Since [A°°] are included in My^^ip^i — 1), p(z); L(i)), the intersection are not empty. The 
transversality condition (jb]) in Proposition 12.81 implies that 

2|L(z)| + 1 < dimMyxM(p(^ - 1),P(^)) 
for each i. From the additivity of the index, we have 

2(/32 -A) + 2 = dim My xr(p, a) 



^ dim MyxR(p(z - l),p(0) 



!=1 

r 



>5^(2|L(.)| + 1) 



=1 



>2(/32-/3i)+r. 
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Therefore we get r < 2. We assumed that r > 1, so r = 2. 

Put L := L(l) Using (jb]), ([c]) in Proposition 12.81 we can easily see that 

[AT] e M^.^ip, p(l); G ,j,(p(l), a; L,\L) 

and that 

dimMO,i,(p,p(l)) =2|L\Li|, dimM° ,k(p(1), ^; M^) = 2|i.2\i^|. 

Conversely for each [A] = ([Ai], [A2]) G M° ^^(P, P(l); A^i) x ^yxM(p(l), A\A, we 
have gluing maps 

: U[A^] X (To, 00) X U[A2] — > M^^^{p,a), 

for some precompact open neighborhoods U[Ai], U[A2] of [Ai], [A2] and positive number Tq > 0. 
When U[Ai], U[A2] are sufficiently small, the transversality conditions in Proposition 12.81 imply 
that intersections 

U[A,]n fl Vi, U[A2]ri fl Vi 

l£L\Li leL2\L 

are transverse and that 

U[A,]n fl Vi = {[A,]}, dU[A,]ri fli^ = 0, UinVi = (li {leL^XL), 

ieL\Li ieL\Li 

U[A2]r\ fl ^^ = {[^2]}, 5f/[A,]n n v^ = 0, f/2ni^ = {ieL\L,). 

ieL2\L ieL2\L 

Put 

s; :=G/(sKp,r)KsKr,a)) : U[a,] x (Tq, cx)) x ?7[^,] ^ Cf{p,a)UG^^ 
and let V/ be the zero locus of s^. Then for each Ti > Tq we have 

X {Ti} X f/[^,]) n f V/ = {i[A^],T,,[A,])}, 

ieL2\Li 

diU[A,] X {TJ X U[A2]) n n ^' = ^ 

«eL2\i^i 

and the first intersection is transverse. The condition ^ in Proposition 12.81 means that s;(p, a) 
and are close to each other on the end of the moduli space. Hence for Ti large enough, the 
intersection 

{U[A,] X m X U[A2]) ri n ^ 

l€L2\Li 

is transverse and consists of a single point which is close to ([Ai],Ti, [^2]). We consider a 
subset 

M' = M^,^{p,a■,L2\L^) \ \J [A2], Ti), 

i[AiUA2]) 

of the moduli space. Here 

E{[A,], [A2],Ti) = Gl[A,UA2]{U[A,] X (Ti,oo) x f/[^,]). 
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Then M' is compact and if we perturb sections si{p, a) outside of neighborhoods of the triples 
([Ai],Ti, [A2]), M' becomes a smooth manifold of dimension 1. Moreover there is a natural 
identification 

(7) 9M'= U U U M°,«(p,r;A^i)xM°,K(r,a;LA^). 

/3 [r] L 

/3i</3</32 <5y([T])= ^^^^51^^ 
j-2/3-1 mod 8 \^\-P 

By counting the number of DM' with signs, we get 

< dd{p ® 7lJ, a ® >= 

and this gives 

dod = Q 

as required. 

Definition 2.12. HFF^{Y;^ := H4CFF,{Y;j),d). 

2.3. Construction of s/(p, cr). In this section, we will prove Proposition 12.81 

First, for flat connections p, cr with dim My ^]g(p, a) < 8, we take locally finite open covers 

{Ux}xeA{p,a) of Myx]R(p, 0") with Ux precompact as follows. We will use a partition of unity 

associated with the open cover to construct admissible sections. 

If dim My ^jg(p, cr) = 0, My^]g(p, a) is compact. Put Uq = My^jg(p, a). Then {Uq} is the 

required open cover. Fix an integer m with 1 < m < 7 and suppose that we have open 

covers {Ux}xeA{p,cT) for fiat connections p, o" with dim My ^]g(p, o") < m — 1. We consider fiat 

connections p, cr with dim My ^rIP; = For each A = (Ai,...,Ar.) G A(p, p(l)) x ■■■ x 

A(p(r — 1), cr), we have a gluing map 

Glx ■■ Ux, X (Ta, 00) X ■ ■ ■ X {Tx, 00) X Ux^ M^^^{p, a) 

for some Tx > 0. We can suppose that Glx extends to the closures Uxi of Ux^- For n = 
(ni, . . . , Ur-i) G {Z>oY~^, we put 

Ux,n ■■= Glx (^Ux, X {Tx + ni,Tx + ni + ^) X ■■■ X {Tx + n^-i, Tx + + ^) x Ux,^ . 

This is a precompact, open set of Myy.^ipjO'). We write for the complement of the union 
of the sets Ux,n- 

Ar:=M°,Jp,a) \ (J 

Lemma 2.13. Under the above notations and assumptions, N is compact. 

Proof. Let {[A"]}^ be a sequence in A^. By Proposition 12. 5[ there is a subsequence {[^4"']}^' 
such that 

[A-']-^{[An...,[Ar])- 
Note that Zi are empty since the dimension of the reduced moduli space is less than 8. If 
r > 1, it follows from Proposition 12.61 that [A" ] lie in the image of Glx for large a' and some 
A. This is a contradiction to the fact that [A"'] are in the complement A^ of the images of the 
gluing maps. 

□ 
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Take a small neighborhood Uq of in My^]g(p, a), and put 

m 

{UxjxeAM ■■= {f/o} U U U U U {f^,!.}- 

r=l p{l),...,p{r~l) A n 

Then {f/A}AeA(p,o-) is a locally finite open cover of My^]g(p, cr) with [/a precompact. We impose 
a condition on T^, where A = (Ai, . . . , A^). If Aj has the form Aj = (A'^, . . . , X'^,;n[, . . . , n'^,_i), 
we suppose that Tx satisfies the inequality 

(8) Tx>J2(Tx'+<) 

k 

where A' = (A'^, . . . , A^,). This condition will be used for the proof of Proposition 12.81 

Next we construct admissible sections s;(p, cr) using a suitable partition of unity associated 
with the open cover {Ux}x- We again do it by induction on dim My (p, o"). Let p and a be 
fiat connections with dim My (p, c") = 0. For generic sections si{p,a) on My^ig(p, a), the 
zero locus are empty, and si{p, a) have the properties in Proposition I2.8[ Fix an integer m with 
1 < m < 7. Assume that for p,a with dim My x]g(p, a) < m — 1 we have sections si{p,a) on 
Myx]R(p, <j) having the properties in Proposition I2.8[ We need to construct admissible sections 
s/(p, cr) for p, cr with dimMy^jg(p, cr) = m. We need some notations and lemmas. 

Take subsets / = {ii, . . . ,is}, J = {ji, ■ ■ ■ ,jt} of Ir-i := {1, ... ,r — 1} with / fl J = 0, 
I U J = Ir-i- Here ii < ■ ■ ■ < is, ji < ■ ■ ■ < it- For n = (rii, . . . , n^-i) G {Z>qY^^ we write 
Hi = (^ii) ■ ■ • 5 ''^jJ) Hj = • • • ) "^jt)) for A = (Ai, . . . , A.,.) we write |A| = r. 

Lemma 2.14. Let I, J be non-empty subsets of Ir~i as above. There is a partition of unity 
{f\}\£Aip,(T) satisfying the following condition. For (A, n) G A(p, cr) with |A| = r, there exists a 
positive integer N = N{X,nj) > depending on A and Uj such that if nj > N for all j ^ J 
then fx,n is identically zero. 

Using the partition of unity, we define sections si{p,a) on My^^ipyO-) by 

m 

(9) Slip, ^) = E E E /A,n Glx{si{p, p(l)) K ■ • ■ K sMr - 1), a)) . 

r=2 p(l),...,p(r-l) A,n. 

Here s;(p, p(l)), . . . , si{p{r — 1), cr) are admissible sections. We will show that we can perturb 
si{p,a) on a compact set in MYy.^{p,cr) such that si{p,a) have the properties in Proposition 

m 

To prove Lemma [2.14[ we need to remove extra open sets from the open cover {Ux}xeA{p,cr)- 
We consider the condition 

(10) Ux,nC U Ux',n'. 

(A',n')eA(p,(7) 

lA'KIAl 

Put 

A' = A'(p,cr) := { (A,n) G A(p,o-) | Ux,n satisfies ([10]) } 
A" = A"(p,^) ■.= A{p,c7)\A'. 
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By definition, {Ux}xeA" is still an open cover of Myxjg(p, cr). Let {fx}xeA" be a partition of 
unity associated with {Ux}\eA"- For A G A(p, a), we define f\ by 

/r ifAeA" 
otherwise. 



fx 



Then {/aIasac^.o-) is a partition of unity associated with {?7a}a(p,o-)- We show that this partition 
of unity has the property of Lemma [2. 141 It is sufficient to show the following: 

Lemma 2.15. Let I, J, (A, n) G A(p, a) be as in Lemma 2.14 Then there is a positive integer 
N = N{\,nj) such that Ux^n satisfies OOj) if rij > N for all j G J. 

Proof. We give the proof in the case A = (Ai,A2,A3), / = {1}, J = {2}. (The proof in the 
general case is similar.) 

For [A] G Ux^n, we can write 

[A] = Glxi[A,],T,,[A,],T2,[A,]). 

Here [Ai] e Ux,, Tx + ni < Ti < Tx + rii + ^. As T2 ^ oo, [A] converges to ([^12], [A3]) 
for some [^12]- Hence there is a positive real number = T2°([Ai], [A2], [A3],Ti) such that 
if T2 > T2 then [A] lies in the image of a gluing map. Therefore [A] is included in Ux',n' 
for some (A',n') G A{p,a) with |A'| = 2 (< |A|). Since Ux„Ux„Ux„{Tx + ni,Tx + Ui + |) 
are precompact, we can take uniformly with respect to ([^i], [^2], [^sj^^i)- Therefore we 
obtain the statement. □ 



Define sections s/(p, cr) by ([9]). Then we have: 

Lemma 2.16. The sections si{p,a) satisfy 0) in Proposition \2.8i . 

The proof will be given in Subsection 12.41 Here we prove that we can perturb the sections 
si{p,a) such that the sections become admissible, assuming Lemma [2.161 We show the next 
lemma to do this. 

Lemma 2.17. For each L C {1, . . . ,d} with dim My^jg(p, a) < 2\L\, the intersection My^^ip, cr; L) 
is compact. 

Proof. If MyxR(p, <J; L) is not compact, there is a sequence {[A"]}^ in Myy^^ip, cr; L) converging 
to some ([Af^], . . . , [A'^]) with r > 1. From Lemma [2.16[ we have 

sii[A''])^sii[A^])m---msi{[A'^]) 

as a — )■ 00. Since Si([A"]) = for / G L, we have 

si{[Ar])^---msi{[Ar]) = o. 

This means that for each / G L, there is some i{l) such that ^^([A^;^]) = 0. Put 

L(z) := { / G L I Sii[Ar]) = }, 

then we obtain 

r 

L = \jm. 

i=l 

Hence we have 

r 

(11) \L\<Y,m\. 

1=1 
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Using this inequality, we show 

(12) dimMO,j,(p(2o - l),p(^o)) < m^o)\ 

for some io- If not, we have 

dimMyxR(p(«- l),p(0) >2|L(^)| + 1 

for all i. From (1111) . 

r r 

dim MyxM(p, ^t) = ^ dim MyxR(p(i-l),p(i)) > 5](2|^(0I + 1) > 2|L| +r > 2|L| + 1. 

1=1 i=l 

This is a contradiction since we assumed that dimMyxjg(p, a) < 2\L\. We have obtained f|T2l) . 

By the hypothesis of induction, si{p{io — l),p(io)) satisfy (jbj) in Proposition I2.8[ Hence the 
inequahty (fT^ means that 

^yxR(p(^o-l),p(^o);^(^o)) = 0. 

On the other hand, [A'^] lies in Myy^^ipiio — 1), p(*o); -^(^o)) by the definition of L{io). We 
have obtained a contradiction. Therefore My^]g(p, a; L) is compact. 

□ 

Proof of Proposition \2.8l 

When dim M^^^{p, a) = 0, generic sections s;(p, a) are admissible. Let m be an integer with 
1 < m < 7 and suppose that we have admissible sections s/(p, a) when dim My^]g(p, a) < m—1. 
Let p, 0" be fiat connections with dim. My^^XP-i^) ~ follows from Lemma 12.161 that 

the sections si{p,a) defined by ^ satisfy (jlj) in Proposition 12.81 From Lemma [2.171 the 
intersections My^]g(p, a; L) are compact if dim My xjg(p, cr) < 2|L|. Hence perturbing si{p,(r) 
on a compact set of My^]g(p, cr), the transversality conditions (jb]), (jcj) in Proposition 12.81 are 
satisfied. Since the region where si{p, a) are perturbed is compact, the perturbed sections also 
satisfy ([a]). Therefore the perturbed sections are admissible. 

2.4. Proof of Lemma 12.161 It remains to show Lemma 12.161 For simplicity of notations, 
we give a proof of the case when [A"] converges to [^2°])- To do this, we show three 

lemmas. 

Let [A"] converge to {[A^], [A'^]) E M^^^{p,p{l)) x ^^{p{l) , a) . In the first lemma, we 
consider the situation where the connections [A"] can be written as gluing of three instantons. 
For example, such a situation occurs if [A°'] are given by 

[A^] = Ghi[A,],T,,[A2],T^,[As]). 

Here A, [Ai], [A2], [A3] and Ti are independent of a, and — t- 00. 
Assume that [A°'] G G/a" with |A"| = 3. We can write 

for some [5f ] G Uxf = 1, 2, 3) and S*" > Ty^ (j = 1,2). The first lemma is the following. 

Lemma 2.18. There is a subsequence {[A"']} a' which satisfies the following. 

(1) )^ are independent of a' . We denote A" by X = (Ai, A2, A3). 

(2) For all i, [Bf] G Ux^ converges to some [-8°°] G [/a, ■ 

(3) We have either Sf < 00, S^' ^ 00 or Sf' 00, S^' ^ <oo. 
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We can show similar statements to this lemma when we assume |A"| = £ with i > 4. As we 
will see later, these statements imply that when [A°'] splits into two instantons there are no 
terms with | A| > 3 in ([9]) for large a . 

In the second lemma, we consider the difference of two gluing maps. Let be a 

sequence in M%^{p,a) converging to some ([Af ], [A^]) G M%^{p, p{l)) x M%M'^),'^)- 
Let Ui, U[ be precompact open neig hborhoods of [Af] in M^^^{p{i - l),p(i)) for i = 1,2. 
(Here p(0) = p, p{2) = a.) Then we have two gluing maps 

Gl:UiX (To, oo) X f/2 ^ M° ^jj(p, a), 

Gl' : U[ X (To, oo) X t/^ M° ,^(p, a). 

When a is sufficiently large, we can write 

[A^] = G/([A?],T", [A^]) = G/'([A'°],T'^ [A',"]) 

for some [A^] G Ui, e Ul, T^^t'^ > Tq. 

Lemma 2.19. Under the above notations, we have 



lim 

o— ^oo 



As before, let {[A"]}q, a sequence of instantons which converges to a pair ([A^], [A^]) G 
M^^^{p,p{l))xM^^^{p{l),a). For A with [A"] G Im G/a and with |A| = 2, we have [A^(A)] G 
U\i, T"(A) > Ta such that 

[A"] = G/a([A?(A)],T"(A),[A?(A)]). 
In the third lemma, we consider the behavior of [A" (A)]. 
Lemma 2.20. 

(1) There is an integer ao > such that if a > ao, then [Af{X)] have the same flat limits as 
[A^]. That IS, [Af{\)] G M^^M^ - l),p{i)) for a > a^. Here p(0) = p, p(2) = a. 

(2) For any positive number 5 > 0, there is some > ao independent of X such that if a > as 
we have 

rf(K(A)],m)<5 

for all X with \X\ = 2, [A"] G ImG^A andi = 1,2. Here d{-, ■) is the metric on My^jg(p(z — 
l),p(i)) induced by the L^^^ -norm. 

Before we prove these lemmas, we show Lemma 12.161 using the lemmas. 

Proof of Lemma \2.1(A assuming Lemma \2.1S\. \2.19[ and \2.2(^ 

Let {A"]}^ be a sequence in My^jg(p, a) with limit ([Af ], [^2°])- Let Ui be precompact open 
neighborhoods of [A°°] for z = 1, 2. Then we have a gluing map Gl from Ui x (Tq, oo) x U2 to 
MyxK(P) For large a, we have 

[A"] = G/([/l?],T^[yl°]) 
for some [Af] E Ui, T" > Tq. We need to show 

si{[A-]) - GIt^ {si{[A^]) m si{[A^])) I = 0. 

First we prove the following claim. 



lim 

a— ^00 
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Claim 2.21. Let {/a}a be the partition of unity as in Lemma [2.141 Then there is a positive 
integer ai > such that if a > ai, then /a^„([A"]) = for all (A, G A(p, cr) with |A| > 3. 

Proof. Otherwise we have a subsequence {[A"']}^/ such that ([^°']) 7^ for some 

(A° ,n°') G A(p, a) with |A° | > 3. From the additivity of index, we have 

I A"' I < dim My xm(p, a). 

Hence we may assume that |A" | are independent of a'. For simplicity, we suppose |A° | = 3. 

By Lemma 12.181 we can suppose that A" are independent of a' and we denote it by A = 
(Ai, A2, A3). Since the supports of /a „c' are included in U^na' and /a„q'([^" ]) 7^ 0, [A" ] lie in 
Hence there are [Bf] G Ux^ for i = 1, 2, 3, and Sf > T\ for j = 1,2 such that 

K'] = GhilBf], Sf, [<], S^', [<]). 

We can assume that [Bf] converge to [B°^] for i = 1, 2, 3 and that Tf — )• < 00, Tf 00 
by Lemma 12.181 The rf are pairs (n" , 77-2 ) with 

Tx + nf <Sf <Tx + nf + 

Hence nf — )■ nf < 00, nf — ?■ 00. Put / = {1}, J = {2}, then we have 

nf >N{X,nf) 

for large a'. Here N{X,nf) is the integer which appeared in Lemma [2.14[ Hence fxn°'' are 
identically zero for large a'. This is a contradiction. □ 

It follows from this claim and that 

(13) sKK]) = h^n-i^) ^A,T"(A) (^KK(A)]) K ^KK(A)])) 

A=(Ai,A2) 
[yl°]elmG/A 

for a > «!. For i = 1,2 and 5 > 0, we put 

f/,, := { [A] G M°,Jp(z - l),p(z)) I [A-]) < S }. 

The open covers {t/A}AGA(p(i-i),p{j)) are locally finite, hence if 6 is sufficiently small then the 
numbers of open sets Ux with Ux H Ui^s non-empty are finite for all i. We fix such 6 > 0. 
By Lemma [2.201 there is a positive integer as such that if a > as, we have [A°'(A)] G t/i,<5 
for all A. Let {Uxi{p)}p=i be the sets of open sets which intersect Ui^s for each i, and put 
a'g := max{ai, as}- Then A in (fT3l) take the form 



A(Pi,P2) = (Ai(pi), A2(P2)) (1 < Pi < qi, 1 < P2 < ^2) 



for a > a'g. 



By Lemma [2.19[ for any e > 0, there exists a{pi,p2] s) such that if a > a{pi,p2]s), we have 



G/t«(sK[A?]) K sK[A^])) - G/a(p„p.),t<^(,i,p.)(3/([A?(Pi,P2)]) K sz([A-(pi,p2)])) 

Here [Af (^1,^2)] = K(A(pi,p2))], T^{pi.P2) = T"(A(pi,p2)). We put 

a{e) := max{ a^, a(pi,p2; £^) I 1 < Pi < gi, 1 < P2 < ^2 }■ 
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Then for a > a{e) 

\\si{[A^])-GlT4sii[A",])^Sii[A^]))\\ < J2 /A,n«(A)(K])^ = ^. 

A=(Ai,A2) 
[A"]&mGlx 

Therefore si{[A"]) converges to si{[A'^]) Kl si{[A'^]). 

□ 

Here we prove Lemma 12.181 and I2.2U[ The proof of Lemma 12.191 will be given in the next 
subsection. 

Proof of Lemma \2.1S\ 

Let [A°'] , [Bf] and S" be as in Lemma 12.181 There is a subsequence { [A" ] }„/ such that 
[Bf] converge to ([-Bf^], . . . , for all i. We will show that = 1 for all i. This implies 

(1) and (2) since the oven covers {Ux}x are locally finite. 

If not, > 1 for some ig- For simplicity, we assume ri = 2, r2 = = 1. Since [Sg ]> [-^3 ] 
converge and {Ux}x are locally finite, we may suppose that A2 , A3 are constant. We write 
A2, A3 for Af', Af'. On the other hand, the sequence {[Bf]}^' has the limit {[B^], [B^])- We 
may suppose that A° take the form A" = (A'^, A2; ) with — )■ 00. Put A' := (A'^^, Ag). We 
assumed that Tx satisfy the inequality (IE]) for all A. Hence we have 

Ty + n"' < Tx^ < Sf 

for j = 1, 2. In particular, S°'' — > 00 for j = 1, 2. This means that [A"] converge to some 

This is a contradiction since [A°'] — )■ ([^1°], [^2°])- Therefore we obtain ri = r2 = = 1. 

Lastly we prove (3). If (3) does not hold, then we may suppose that 5*" — j- S°° < 00 
for j = 1,2 or Sf — )■ 00 for j = 1,2. In the first case, [A"''] does not split into instantons 
as a' — 7- 00. In the second case, [A"''] splits into three instantons as a' — t- 00. Both cases 
contradict the fact that [A"] split into two instantons. 

□ 

Proof of Lemma \2.2(A 

Let [A"], [Af] and T"(A) be as above. Suppose that (1) does not hold. Then we have a 
subsequence and A"' e A(p,p"'(l)) x A(p"'(1),ct) with G ImG/^„' and p"'(l) ^ 

p(l). By Hypothesis 12. H the set of gauge equivalence classes of flat connections is finite. Hence 
we can suppose (1) are independent of a' . We write p'{l) for (1). As in the proof of 
Lemma 12. 18t we can see that there is a subsequence (still denoted by [v4°']) such that [A"'(A° )] 
converge to some A" are independent of a' and that T"'(A" ) diverges to 00 . This means 

that 

[^1 ([^'r], [^2^]) e M°,Jp,p'(l)) X M°,Jp'(l),a) p'(l) ^ p(l). 

This is a contradiction to the fact that [A"] converge to ([Af ], [A^]) e M^^^{p,p{l)) x 
Myx]g(p(l), 0") and we have shown that (1) holds. 

If (2) does not hold, there is a subsequence {[A"']}^' and A°' G A(p, p(l)) x A(p(l),cr) with 

(14) rf([Af(A"')],m)>5. 
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As before, we can deduce that [Af (A" )] converge to some [A°°] for z = 1, 2, A" are independent 
of a', and that T"'(A° ) diverges to oo. Hence [A"'] converge to ([^4']^^], [^2°°]). From (HM . we 
have d{[A°°], > 6. In particular, [Af"] ^ [v4^°°]. This is a contradiction. 

2.5. Evaluation of difference of two gluing maps. In this subsection, we prove Lemma 
12.191 We need to compare two gluing maps defined on different regions intersecting each 
other. To do this, we must explicitly see the construction of gluing maps. We give outline of 
the construction following [8j before we prove Lemma 12.191 

First note that elements in My^jj(p, a) can be considered as elements in Myyj^^p.a) with 
center of mass 0. Here the center of mass of \A\ G MyxR(p, cr) is defined to be 



Yxl 



Yxl 



B, 



Let Xi,X2 be two copies of F x M and for T > put Xi{T) = Y x (-cx),T), XiiY) = 
Y X (— T, oo). For each T, we have an identification 

ip: F X (T, 2T) — > Fx {-2T, -T) 
{y,t) ^ iy,t-3T). 

Gluing Xi(2T) and X2(2T) through ip, we obtain a manifold = Xi(2T) U<^X2(2T). We 

have a natural identification between X*^^^ and F x M such that (y, 3T/2) G Xi(2T) C X*^'^^ 
corresponds to {y, 0) G F x M. 

Let p = p(0), p(l), cr = p(2) be flat connections and choose [Ai] G M^^(p, p(l)), [^2] G 
Mx^ipi^), c"). Here Ai, A2 are instantons on Xi, X2 with center of mass 0. We write p(0), p(2) 
for p, a respectively. Then we can write 

Ai = Bi + Cj, 

where Bi are connections with 

p{i — 1) on F X (—00, —1) , 
p{i) on F X (1, 00) , 

and Oj are 7r*gQ-valued 1-forms on Xi with 

\V''a,{y,t)\<Cke-'' (^A; G Z>o) 

for some > 0, S > 0. Fix a smooth cut-off function x : F x R — > [0, 1] such that 

y(y t)=l ^ on Fx (-00,0), 

^^^' ^ \ on F X (1,00). 

We define A[, A'^ by 

a; = 5i + xit - T)ai, A'2 = B2 + xi-t - T)a2. 
These connections give a connection A' on X**^-^^ = F x R such that 

r cUT/2i^[) onFx (-00,-f + 1] 
(15) A'=l p(l) onFx(-f + l,f-l) 

[ c^T/2K) on F X [f - 1,00) . 

Let be the self-dual part of the curvature of the connection on vr*0Q induced by A'. Then 
we have 

II^A'llLl(yxR) < const e"^^. 
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That is, A' is an almost instanton. To obtain a genuine instanton, we consider the equation 

for a small 7r*gQ-valued 1-form a. The equation can be rewritten as 
(16) d\,a + (a A a)+ = -Fj. 



By Hypothesis \2.2\ D^^ = + a^, have right inverses Pi. When T is large enough, we can 
construct a right inverse P = Pt of D^' by using Pi,P2 and some cut-off functions. We can 
see the operator norm ||P|| of P is bounded by 2(||Pi|| + ||P2||). If we substitute a = Pep into 
( 1TB]) . we get the equation 

+ (P(/) A P0)+ = -F+ 

for (j) G L\{A.Yy^^ ® t^*Qq)- We can see that for large T > this equation has a unique solution 
(j) = (pT with — ^ as T — )■ oo. Hence for large T we get an instanton of the form 

(17) = A' + a 
with 

(18) lim a = 0. 

T-^oo 

For f/j C Mx^{p{i — l),p{i)) precompact open sets, the construction can be applied to all 
{[Ai], [A2]) G f/i X f/2 and we obtain the gluing map 

Gl:UiX (To, 00) X f/2 ^ M° ^k(p, ^) 

for large Tq. (We can take Tq uniformly since Ui, U2 are precompact.) 

More precisely, we must slightly translate A*^"^^ to make the center of mass be 0. Let uit 
be the center of mass of A*^^\ Then the translate c*_^^{A*^'^^) is an instanton with center of 
mass 0. The precise definition of the gluing map is 

Gl{[A,],T, [A2]) = K^^{A*(^^)]. 

We can easily show 

lim rriT = 

and it does not matter even if we assume that m-r = in the proof of Lemma I2.19[ Therefore 
we will drop cl^^ from notations in the proof. 

Let 7 = S*^ be a loop in Y and put F = 7 x M. We see outline of the construction of the 
gluing map 

Gl : Cf{p,p{l))\u,^Cf\p{l),a)\u, A Cf{p,a)\,^Gi 

covering Gl. Let A^^A^ be instantons on Xi,X2. For simplicity of notations, we suppose that 
are surjective for i = 1,2. Let Fi, F2 be copies of F and fix cut-off functions 7^ = ■yf on Fj 
with 

||rf7.||co = ^ (^) ' supp(7.) C Fi(2T), (7i)2 + (72)^ = 1 on F#(^). 

Here supp(7j) is the support of 7^, and Fj(2T), F**^-^) are defined as before. In the third 
equation, we consider 7j as functions on F**^-^) in the natural way. Take fi E ker Ba^ for 2 = 1,2 
and put 

(19) /' = C„3T/2(7l/l) + C3T/2(72/2)- 
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Then we can show that 

l|<9A#m/'i|^2,{-r,r) < £:(T)(||/l||^2,(-r,r) + 1 1 /2 1 1 ^2. (-r,r) ) , 

e{T) — ^ as T ^ oo. 

We may construct a right inverse Qt from right inverses Qi of Ba, for large T. The difference 

(20) f*i^) = f'-Q^dA*mf' 

lies in ker9A#(T). The operator norm \\Qt\\ of Qt is bounded by 2(||Qi|| + IIQ2II), hence we 
have 

(21) QTdA*mf'^0 

as T — )■ 00. It can be shown that for large T the map (/i, /2) H- /#(^) is an isomorphism from 
ker ^^^©ker to ker 8^*^ and induces the isomorphism from C^Xp^ p(X))[Ai]®i^Ti {piX)^ ^)[A2\ 
to CyXp^^)[a*^t)y Applying this construction to all connections in Ui x f/2, we get the map 

Gl. 

When are surjective, we choose maps 

Ui : M"" — y n'^^'^iE (g) Kl) 

such that dAi ® are surjective. Applying the above method to these operators, we obtain 
the gluing maps. 
Proof of Lemma \2.19[ 

Assume that we have a sequence {[A^jjo, in My^^ip, o") which converges to some ([^4^], [A^]) e 
My^]g(p, p(l)) X My^]jj(p(l), 0"). Take precompact, open neighborhoods Ui,Ul of [A°°]. Then 
we have two gluing maps 

Gl■.U^x (To, 00) X [/2 M° ,j,(p, a) 

Gl' : U[ X (To, 00) xU!,-^ M^^^ip, a). 

For large a, we have two different expressions of [A°']: 

= G/(K],T^ [A^]) = Gl'{[A[%T'-, [A',-]). 

We can take representations Af, A^'',_Af of [Af], [A^"], such that A" converge to 
For simplicity, we assume that dAi surjective for all [Ai\ G Ui. Then what we must 
show is that 

lim ||G/T^(/r, f^) - Gl'^.M[-, /;")||l^.(-...) = 

a— ^00 

for sequences {/f }a, of ker (9a«, kerS.'c which converge to some /°° G ker5yi«> in the 

Lg'^'^'^^-norms. From ([HD, ([20]) and ([21]), we have 

G'/T"(/r, A") = c_3Tf,(7i/r) + C3T^(72/2") + ^7" 

fool ^ ^ 

^ ^ G/^'.(/;°, /;") = C ar:. (71/;") + C,Ml2f'2'') + ^7'" 

2 2 

for some sections (7°, with g°',g'°' — 0. Here we show that the difference T" — T'" goes to 
zero as a — )■ 00. 

Lemma 2.22. lim„_,„o |T" - T'°| = 0. 
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Proof. Since [A°'] converges to ([^4^], [v4^]), we have real numbers S^, S2 with 5*2 — 5*" — )■ cxd 
such that for any compact sets K in Y x W, 

(23) cs^{[A-\k]) ^ [ATIk] 

m L4' . 

On the other hand, [A"] can be written as Gl{[A'^],T'', [A^]). It follows from (fTSD. ffT?!) and 
18]) that 



(24) c^_^i[A''\K]) ^ [ATIk]. 

Comparing (1251) with fl24l) . we have 

lim |5i"-(3T"/2)| = 0. 

a— >oo 

Similarly we have 

lim |Sf - (3r''^/2)| = 0. 
Hence we obtain the required result. 

□ 

From fl22l) . we get 

< iic_3T^,(7i/r) -c_3T^(7i/;")ii + iic_3T^(7i/;") -c_3^(7i/;")ii+ 

Heap (^2/2") -C3r^(72/;")|| + ||C3T^(72/;") -C3^(72/;°)|| + + \\g'% 

Here || ■ || is the L^'^""^'^) norm. The first term on the right hand side is equal to 

ii7i(/r-/;")ii. 

This is bounded by || /{* — || , and || /f — || converges to zero since both of /f , converge 
to f^. Hence the first term goes to zero. 

The second term is equal to 

ii7i/;"-c,.(7i/;°)ii, 

where 5° = 3(T" - T°)/2. It follows from Lemma [222] that 5" goes to zero. Since is 
finite and {5"}q- is bounded, for any £ > there exist T{e) > independent of a such that 

WfrWxi-oo, -T{e))\\ < g, ||/r|yx(T(£), oo)|| < g 

l|c<5"(/r')|yx(-oo, -t(e))|| < g, ||c5"(/r')|yx(r(£), 00) || < g- 
Since converge to f^, we have Q;(e) > such that if a > a{e) then 

ii/i^-zrii <!• 
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Thus we have 

(7i/;'^-c^.(7i/;")) ^ ^ ^^^^ 

< ||7l/l"|yx(-oo, -T(e))|| + ||c5a(7l/i")|yx(-oo, -T(£))|| 

< ll/i" - /ni + ||/r'|yx(-oo, -r(£))|| + ||c5«(/i" - /Dll + ||c5«(/r')|yx(-oo, -T(£))| 

< -. 



Similarly 

(7i/;"-c,.(7i/;"))L^^^^^ ^ 

Yx(T(s), oo) 

Since 7i/{°° is uniformly continuous on y x [—T{e),T{e)] and 5"^ converge to zero, we have 



e 

<2- 



(7i/r-c..(7i/r)) 



yx[-T(e),T{e)] 



< 2T(£)M(£) raax 17^^ - C5<^(7i/r)l ^ 

Yx[-T{e),T{6)] 

as a — )• OO. Here M(£) is the maximum of the weight function over Y x [—T{e),T{e)]. 
Hence we get 



< 



(7i/;" - 7i/r) 



Yx[-T{e),T{s)] 
Yxl-T{e),Tie)] 



Yx[-T{6),T{e)] 







as a ^ oo. Therefore 

limsup ||7i/i" - C5«(7i/i") 



< limsup 

a—^oo 



(7i/;"-c5.(7i/;")) 
(7i/;"-c5«(7i/;")) 



yx(-r(e), T(£)) 



yx(-oo, -r(£)) 



Yx{T{s), oo) 



Since e is arbitrary, the norm of difference ||7i/i" — C5c»(7i/{")|| goes to zero. Thus the limit 
of the second term is zero. Similarly the limits of the third and fourth terms are zero. Since 
ga g'a^Q^ we have 

hm WGlMf^, f^) - /2")ll = 

a— >oo 

as required. 

2.6. Well-definedness. A priori, Fukaya-Floer homology groups seem to depends on the 
choices of Riemannian metric on Y and sections of /lf'^{p, a). But we will prove that Fukaya- 
Floer homology does not depend on these choices up to canonical isomorphism. In the proof 
of well-definedness of usual Floer homology groups, we need the functorial property of Floer 
homology groups with respect to Riemannian bordisms. We shall generalize the functorial 
property to Fukaya-Floer homology groups and prove the well-definedness by using this prop- 
erty. 
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The main statement of this subsection is 

Proposition 2.23. Take two Riemannian metrics go,gi on Y and two sets {si{p,(j)}p^^j, 
{s'i{p, <j)}p^cT,i of admissible sections. We write HFF^{Y] 7), HFFl{Y; 7) for Fukaya-Floer ho- 
mology groups associated with the metrics and sections. Then we have a canonical isomorphism 
between HFF^{Y;j) and HFFl{Y]j). 

Assume that we have the following data: 

• two oriented, closed Riemannian 3-manifolds (Y'ojS'o)) iXiigi)i 

• [/(2)-bundles Qq.Qi over Yq,Yi satisfying Hypothesis I2.H 

• sets of loops 7 = {7«}f=i, y = {7i}f=i, 

• a Riemannian bordism (X, G) between (Foj^'o)) (Xiigi)-, 

• a f/(2)-bundle P over X with P|y„ = Qo, ^In = Qi, 

• oriented, compact surfaces embedded in X with boundary 7; ]J 7^. 
We introduce the following notations: 

• X := X U (Fo X M>o) U (Fi x M>o), 

• the extension P of P to X, 

• the extension G of G to X, 

• ti := U (7i X M>o) U (7^ x M>o). 

Let p, a be flat connections on Qq, Qi respectively. We denote the moduli space of instantons 
on P with limits p,a by Mj^{p,a). Using families of twisted d operators on E;, we get the 
line bundles C'?^{p,a) over M^(p, a). As the proof of Proposition 12.81 we can show that for 

p,a with dimM;^(p, cr) < 8 there are sections si{p,a) of C'?^{p,a) satisfying the following 
conditions: 

• Let { [A"] }a be a sequence in (p, a) converging to 

([An,...,m,[A°°],[A'r],...,K?°]), 

where 

[Ar]eMl,M^-i),p{t)), 

[A-]GM^(p(r),p'(0)), 

[^heM°^,i,(p'(z-l),p'W). 

Then we have 

siilA'^]) siilA^]) K ■ • ■ K siilA'^]) K siilA"^]) K s[{[A'n) K ■ ■ ■ K s[i[A':r]). 

• For L C {1, . . . ,d} with dim M^(p, a) < 2|L|, the intersection Mj^{p, a; L) = Mj^{p, a)r\ 
Hk^lVi is empty. 

• For L with dim ilP^ (p, a) = 2|L|, the intersection Mj^{p,a; L) is transverse and com- 
pact. Hence the intersection is finite. 
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Lemma 2.24. Using the data (X, G, s;), we can define a homomorphism 

C : CFF,{Yo;2) ^ CFF,{Y,-j;) 

with the following property: 

• The homomorphism ( is a chain map. That is, dio ( = ( o Oq. 

• The induced homomorphism : HFF^{Yq;'^) — )■ ifFF*(Fi;7') is independent of the 
metric G and the sections si{p,cr). 

• Assume that we have a bordism {X', G', P', {^'i}f=i) from (Yi, gi, Qi, 7') to other 4-tuple 
(^2,c/2,<52,7")- Let C be the map from GFF{Yi;i) to GFF{Y2;Y). For T > we 
define a bordism X*^^) between Xq, X2 to be 

X(^) := XU (Fi X [0,r]) UX'. 

The metrics G, G' naturally induce a metric G^'^^ on X^'^^ and we have surfaces = 
Si U 7' X [0, T] U SJ with boundary 7; ]J 7". 5*0 we have the map C^^) from GFF,{Yo;-f) 
to GFF^iY2]i') . Then for large T > 0, 

We can derive Prop osition l2.23l from Lemma [2.241 Take two Riemannian metric gQ,gi on Y 
and two sets {si{p, cr)}i, {sj(p, a)}i of sections of Cf'^{p, a). Put X = F x [0, 1], = 7; x [0, 1], 
P = 7i*Q. Here tt is the projection from X = Y x [0, 1] to Y. Choose a Riemannian metric G 
on X with G\yo = go, G\y^ = Qi- These induce the maps 

C.: HFF^{Y-^ HFF'SY-j), 

C:HFF:{Y;j)~^HFF^iY;j). 

On the other hand, the metric Gq = go + dt^ and the pull-backs p*{si{p,a)) of si{p,a) by 
p : MyxK.{p, o") ^y-xr(P' ^) induce an endomorphism ^'^ of HFF^{Y] 7). It follows from the 
construction of C, we will see below that is the identity map. The third part of Lemma [2.241 
implies 

Similarly the composition C* oC is the identity map of HFFliY; 7). Thus C* is an isomorphism 
from HFF^iY; 7) to HFFi{Y; 7). 

We begin the proof of Lemma 12.241 Recall that the degree 5yo([p])5 defined by 

^Yoiip]) = indL'ACp.po) mod 8, 
^yAW]) = ind DA(a,pi) mod 8 
for some fixed flat connections po, pi over Yq,Yi. For simplicity of notations, we suppose that 

ind Da(po,p,) =0 mod 8. 

Here A{p, a) is a connection on X with limits Pq, pi. Then the dimension of Mj^{p, a) is equal 
to (5yi(cr) — (5yo(p) modulo 8. 

For < /So < A ^ d, Lq C Li C {l,...,d} with |Lo| = (3o, \Li\ = f3i and generators 
[p] G CFj_2/3o(Fo), [cr] G CFj_2;3i(Fi), the intersection Mj^{p,a; Li\Lq) is finite. We define 

<C([p]®7Lj,M®7i, >by 

< cap] ® 7lo), M ® 7li >= #^x(P. ^i\^o)- 
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Then the matrix elements give the map ( : CFF^(Yq; 7) — )■ CFF^(Yi; 7'). That is, 

/32 H Li 

We give outline of the proof that ( has the properties stated in Lemma 12.241 

The first part follows from a similar discussion to that in the proof that d o d = 0. Take a 
generator [r] G CFj_2i3i-i{yi)- Then we have a cut-down moduh space Mj^{p,T; Li\Lq) with 
dimension 1. Counting the number of the end of this moduli space, we get 

5i o C = C o <9o- 

To prove the second part, take other metric G' and sections s'i{p, a) over X. Then we have 
another map (' : CFF^(Yq; 7) — > CFF*(Yi, ; 7'). We will construct a chain homotopy 

H : CFF,(Fo;7) — ^ 7') 

such that 

(25) C' -C = H odo + dioH. 

Take a path {G'*}o<s<i of metrics on X from G to G', and put 

M(p,a) := U M^ ^.(p,(t) x {s}. 

0<s<l 

Then M(p, a) is smooth for generic paths of metrics. We can define line bundles Cf'^{p, a) over 
M(p, cr) as usual. As in Subsection 12.31 for p, cr with dim M(p, a) < 8, we can construct sections 
si{p, cr) such that s(p, a) are compatible with gluing maps, {si{p, a)}i satisfy the transversality 
conditions as in Proposition 12.81 and 

Si(P,0-)|M^^o(p,'T)x{0} = Si{p,(t), S/(p,Cr)|M^_^i(p,<T)x{l} = sJ(p,Ct). 

For < (3q < f3i < d, Lq C Li C {!,..., d} with |Lo| = f3o, \Li\ = (3i and generators 
[p] G CFj_2/3o(^o)) W] £ C*Fj_2/3i+i(yi), we have a moduli space M{p,a) with dimension 
2(/3i — /3o)- Cutting down M{p,a) by the sections, we have the finite set M{p,a; Li\Lo) = 
M{p, cr) n riieLiVLo ^i- Here is the zero locus of si{p, a). We put 

< H{[p] ® 7lJ, [a] ® 7L, >:= #M(p, a; L^Lo). 

These matrix elements give H : CFFj{Yo;'j) — i- CFFj+i(Fi; 7') as usual. We can show the 
chain homotopy condition ( l25l) by counting the number of the ends of the cut-down moduli 
spaces M{p,T] Li\Lq) for generators [r] G CFj_2i3-i^{Yi). Thus we obtain the second part of 
Lemma [2^^ 

The proof of the third part is essentially same as the proof of the gluing formula for Donaldson 
invariants which will be given in the next subsection, and we omit the proof here. 

We give a remark on the dependence of HFF^,{Y;'~f) on 7. It seems that HFF^,{Y]'y) 
depends only on the homology classes [7/] G HiiY; Z). If •ji and 7^' are homologous, then we 
have oriented, compact, surfaces E/ in y x [0,1] with boundary 7/II7;. As above we can 
construct a linear map 

Cs:CFF,(r;7)^CFF,(F;7') 
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where S = (Si,...,Sd). To prove that induces an isomorphism from HFF^{Y;'y) to 
HFF^{Y;'y'), it is sufficient to show that the map has the properties as in Lemma [2.241 
The most difficult part of the proof is to prove that the induced map between the Fukaya-Floer 
homology groups is independent of S. 

Suppose that we have another bordism SJ from 7^ to 7^. We would like to show that the 
two maps between the Fukaya-Floer homology groups induced by and C^' ^ire the same. A 
natural way to prove this is to use an isomorphism between (p, a) and £^;^ (p, a) which is 
compatible with the gluing maps in an appropriate sense, if the isomorphism exists. Hence the 
problem reduces to the existence of the isomorphism. This can be regarded as a generalization 
of the fact that the index of a family of elliptic differential operators on a closed manifold 
depends only on the bordism class. 

We can take a compact, oriented manifold W with corner, whose boundary is 

(26) S,US;u(7iX [0,l])U(7;x [0,1]) 

as follows. Since [7;] is equal to [7^'] in Hi(Y; Z), there is a complex line bundle L over Y and 
sections s, s' of L with s~"'^(0) = 7;, s~^(0) = 7^. Moreover there are sections s, s' of the 
complex line bundle L x [0, 1] over Y x [0, 1] with 

S|yx{o} = s, 5|yx{i} = s', 5"^(0) = S;, 

5'|yx{o} = s, 5'|yx{i} = s', s'^{0) = S,. 

Take a section s of the line bundle L x [0, 1] x [0, 1] over Y x [0, 1] x [0, 1] with 

•s|yx[0,l]x{0} = 5, s|yx[0,l]x{l} = 5', s|yx{0}x{M} = S, s\yx{1}x{u} = s' (u G [0, 1]). 

For a generic s, the zero locus of s is a compact, oriented manifold whose boundary is fl26l) . 
Attaching 7; x M>o x [0, 1], 7; x R>o x [0, 1] to W, we get a "bordism" W from to SJ, i.e. 
W is an oriented manifold with boundary ]J S^. 

If we mimic the proof of the invariance of the index of operators over closed manifolds with 
respect to bordism, it may be possible to prove that there is an isomorphism from Cj.^{p, a) to 
Cf., (p, cr) which is compatible with the gluing maps. However, we do not discuss this in this 
paper and we just state it as a conjecture. 

Conjecture 2.25. In the above notations, the bordism W between S/ and gives a natural 
isomorphism between £g (p? cr) and Cf^,{p,cr) which are compatible with the gluing maps. 

If this is true, HFF^{Y; 7) depends only on the homology classes [7^] G Hi(Y] Z). 

2.7. Relative Donaldson invariants. In this subsection, we will see that the Fukaya-Floer 
homology groups allow us to extend the Donaldson invariants for closed 4-manifolds to 4- 
manifolds with boundary. 

Let F be a closed, oriented 3- manifold and 7 = {7z}f=i be a set of loops in Y , where < (i < 
3. For compact, oriented 4-manifolds Xq with boundary F, we write Xq for Xq U (F x M>o). 
We also denote E; U (7^ x M>o) by for surfaces E; embedded in Xq with boundary 7;. Assume 
that Xq is simply connected and ft"*" > 1. Moreover suppose that we have a [/(2)-bundle Pq 
on Xq such that the restriction Q = Pq\y satisfies Hypothesis 12. 1[ Take a metric Qq on Xq 
such that the restriction of ^0 to F x R>o is equal to gy + dt"^ for some metric gy on F. Using 
families of twisted d operators over E^, we can define complex line bundles Cf'^{p) over the 
moduli space (p) of instantons on the extension Pq of Pq to Xq with limit p. As before. 
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we can show that for p with dim M^^(p) < 8 there are sections s/(p) of the hne bundles with 
properties similar to those in Proposition 12.81 

Let po be the fixed flat connection used to define 5y. For simplicity of notations, we suppose 
that 

indDA(po) = mod 8, 
where A(po) is a connection on Pq with limit po- Then we have 

dimM^^(p) = -5y([p]) mod 8. 

Under these hypotheses, we define an element ipxo = V^Xo ^ CFFq{Y] 7) as follows. Let {3 be an 
integer with < /3 < and take L C {1, . . . ,d} with \L\ = {3 and a generator [p] G CF_2/3(y). 
Then we get a number 

< V-Xo, [p] ®7L >:= #M^^(p; L). 
Here M^^(p;L) is the 0-dimensional cut-down moduh space by the sections {s;(p)};gi. We 
define ipxo by 

:= 5^ 5^ 5^ < V^Xo, [p] ® 7L > [P] ® 7L e C'FFo(F; 7). 

/3 L [p] 

The following proposition follows from a combination of a discussion of the case when Y is 
a homology 3-sphere and techniques we have developed in this paper. 

Proposition 2.26. The chain ipxa is a cycle, and the class '^Xo = ^Xo(Si) ■ ■ ■ 1 ^d) £ HFFq{Y; 7) 
represented by ipxo is independent of the metric and the sections. 

Remark 2.27. It seems that \['xo(Si5 • • • > ^d) depend only on the homology classes [Si], . . . , G 
H2{Xq, Y; Z). This is true if an analogy of Conjecture 12.251 holds for the line bundles Cf^i^p). 

2.8. Gluing formula. We consider a situation where a closed, oriented 4-manifold X is cut 
into two parts Xq, Xi along a closed 3-manifold Y . Let P be a f/(2)-bundle over X such that 
the restriction Q = P\y satisfies Hypothesis 12.11 and dim Mp = 2d for some integer d with 
< (i < 3. We suppose that Xo,Xi are simply connected and 6"*" of Xq^Xi are larger than 
1. Let El, . . . , Erf be embedded surfaces in X such that the intersections 7; := F fl S; are 
diffeomorphic to . We denote Xq fl S;, Xi fl by SJ, SJ'. Then we have relative invariants 

vl>Xo(S;, . . . , S:,) G HFF,{Y- 7), vl>^,(S;', . . . , S'j) G HFF,a^,{Y- 7). 

We will express the invariant \E'x([Si], . . . , [SJ) of the closed manifold X in terms of the 
relative invariants Xo-, ^ Xi- To do this, we define a pairing 

< , >: HFF,{Y;j) ® HFF^j^s+2d{Y;x) Q 

as follows. 

For fiat connections p, we have 

Sy{[p]) = -Sy{[p])-3 mod 8. 

Hence CFj{Y) and CF^j^^iY) are dual to each other. We define a pairing 

< , >: CFFj{Y;2) ® CPP_,-_3+2d(>^; 7) ^ Q 

by setting [p]®7lc g CP-_j_3+2/3('^) ®Q < 7l= > as the dual element of [p]®7l ^ CFj^2i3{Y)® 
Q < 7l >• Here L is a subset of {1, ... , d} with |L| = /3 and L'^ is the complement of L. It is 
easy to see the following. 



Lemma 2.28. < d{[p] O 7^), [a] 7^/ >= ± < [p] ® 7^, ^([(t] ® 7^,) > 
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Therefore we get the pairing 

< >: HFFj{Y-j)®HFF_,_^+2d{Y;j) ^ Q 
on Fukaya-Floer homology groups. 
Theorem 2.29. In the above situation, we have 

^xmi . . . , [S,]) = ^ < ^xAK. • • • , ^'d). . . . , S'j) > . 

To prove the gluing formula, we need sections of £f which are compatible with gluing maps 
as usual. Fix a Riemannian metric gy on Y and choose Riemannian metrics 5'o,5'i on Xo,Xi 
such that the restrictions to Y are equal to gy- For each T > 0, we define a manifold 
by X**^"^) = Xq U y X [0, T] U Xi. Then go^gi naturally induce a Riemannian metric f?**-'^^ 
on X#(^). Fix a sequence — oo and we write X" for X'^^'^°'\ As in Subsection 12.31 we 
can construct sections of £f^^ — )■ M^c- such that for each sequence [A"] e Mx« converging 
to ([Ag°], . . . , [A^]) the values Si([A°]) at [A°] converge to sz([A~]) K ■■■ K s,([A~]) in an 
appropriate sense. By dimension counting, we can show that if [A"] lie in the intersection 

d 

n Pi v; 

1=1 

then r = 1 and ([A§°], [A~]) G M^^(p; L) x M^^(p; L^). Here [p] G CF_2/3(r) with < < (i 
and L is a subset of {1,. . . ,d} with \L\ = /3. The transversality conditions for the sections 
imply that M-^^ (p; L) and M^^ (p; L'^) are finite sets. From an argument like that in the proof 
of ([7j), we can show there is a natural identification 

d d 

Mx. n fl = U U U (P; L) X M^^ (p; L'^) 

1=1 13=0 5y{[p])= |L|=/3 

-2/3 mod 8 

for large a. Counting the number of elements of both sides with signs, we obtain the gluing 
formula. 

3. Floer homology for 2-torsion instanton invariants 

In this section, we consider a variant \E'^^ of Donaldson invariants for non-spin 4-manifolds 
X, which is a linear function 

^x' ■ A'{X) Z2. 
Here A'(X) is the subspace of Q)d>oH2{X; Z)®°' generated by the set 

{ [Si] ® • ■ ■ ® [Sd] I [Si] G H2iX; Z), [S^] • [S^] =0 mod 2 }. 

This invariant is defined by a 2-torsion cohomology class Ui of the moduli spaces. Originally 
this was defined for spin manifolds by Fintushel-Stern jTU], and the author extended to non- 
spin 4-manifolds in [T^]. In this section, we construct a variant of Floer homology group and 
prove a gluing formula for . 

In [19], it was shown that is non-trivial for X = CPg#CP?#CP^ where CP^ and CPf 
are copies of CP^. More precisely 

(27) {-Ho + E,Hi~ E) = 1 mod 2. 
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Here Hq, Hi and E are the generators of i/2(CP^; Z), i/2(CP?; Z) and H2{CF'^; Z) respectively. 
We can consider X = CPg#CP?#CF2 as a connected sum of Xq = CPg and Xi = CP?#CP2. 
Since the only fiat connection over 5*^ is the trivial one, we can deduce from fl27|) that the 
trivial fiat connection has an important role in the gluing formula for in contrast to the 
case of usual Donaldson invariants. Note also that the homology class —Hi + E in ( 1271) is a 
sum of homology classes —Hi and E of Xq and Xi with self-intersection numbers odd. 

On the other hand, for closed, simply connected, non-spin 4-manifolds Xo,Xi with b"^ pos- 
itive, we can show the following vanishing theorem. For homology classes [S^] G H2{Xo;Z), 
[Sy G H2{Xi;Z) with self-intersection numbers even, 

(28) no#Xi(Pi], • • • , Pdo], [S'l], . . . , K.]) = mod 2. 

This follows from a standard dimension-counting argument. The formula (l28|l implies that 
the key of the non-vanishing result (!27|) is that the homology class — -ffi + is split into two 
homology classes of Xo,Xi with self-intersection numbers odd. 

In this section, we consider a situation where X has a decomposition X = Xq Uy Xi for 
a homology 3-sphere Y and a closed, oriented surface E in X is split into two surfaces with 
self-intersection numbers odd along Y. We will give a gluing formula for ^^^([E]) in terms of 
differential-topological date about Xq,Xi. This situation is similar to that of Section [2] and 
we can apply the method developed in Section [2J The main difference is that a [/(2)-bundle 
over a homology 3-sphere has the trivial fiat connection, which is reducible, and we need to 
pay attention to the effect of the trivial fiat connection on the gluing formula. 

3.1. 2-torsion instanton invariants for closed manifolds. We will summarize the con- 
struction of 2-torsion instanton invariants for closed non-spin 4-manifolds which is given in 
dg. See also 

Let X be a closed, oriented, simply connected, non-spin 4- manifold with > 1. Take a 
C/(2)-bundle P over X with W2{P) = W2{X). For [S] G H2{X;'Z), we have the complex line 
bundle 

We can show that if [S] ■ [S] = mod 2 then the center {±1} of SU{2) acts trivially on the 
line bundle. Hence we get a line bundle 

£s = Cj:/S0{3) B*^. 

Let c be a spin-c structure of X. For each connections A on P, we have the twisted Dirac 
operator 

flA:T{S+^E) ^T{S~^E). 

Here E is the rank two complex vector bundle associated with P and are the spinor bundles. 
If ci(detc) = — ci(P), we have a "real part" of the Dirac operators: 

C^Ah : r((5+ ® E)u) ^ T{{S- ® 

(See [1].) The family of real operators {{'?'A)M.}[A]eBp gi^^^s a real line bundle 

We also suppose that C2{P) = mod 2. Then we can see that the center {±1} of SU{2) acts 
trivially on A, and we obtain a real line bundle 

A = A/S0{3) A B*p. 
We define Ui G H\B*p] Z2) to be Wi{A). 
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When b'^{X) is even, the virtual dimension of the moduh space Mp is odd. We can write 
dim Mp = 2d + 1 for some integer d. Assume d > 0. Then we define 

. . . , [S,]) = " < ^ii U ci(£sj U ■ • ■ U ci(£sj, [Mp] > " G Z2. 

Here [Si], . . . , [EJ G H2{X; Z) with [Sj] • [Sj] = mod 2. In general Mp is not compact, 
however we can define the pairing as follows. Let ss. : B^. — )• C■s^ be sections and Vs^ be the 
zero loci. We can prove the following lemma by a standard dimension counting argument. 

Lemma 3.1. Assume that the dimension of Mp is 2d + r for some d > and < r < 3. 
Then the intersection Mp fl V^i fl ■ ■ ■ fl V^^ is compact and smooth for generic sections ss . . 

When dim Mp is 2(i + 1, the intersection 

Mp n n ■ ■ • n v^^ 

is a compact, smooth manifold of dimension L Precisely the invariant \l'^^([Si], . . . , [S^j]) is 
defined to be 

< Ml, [Mp n v^Ei n ■ ■ ■ n VsJ >e Z2. 

We can show that \E'^^([Ei], . . . , [T,d]) is independent of the metric on X and sections of C-e^, 
and hence it is a differential-topological invariant of X. 

Remark 3.2. When we define the invariant \l/^^([Si], . . . , [Sj), we use the pull-back r^. (ssj 
of the section sy.. over B^.. Here r^- is the map Mp — )■ ^B^. defined by restricting connections 
to Sj. However Lemma 13.11 shows that when the dimension of Mp is 2(i + 1 with d > 
Mp n Vj]^ n ■ ■ ■ n Vs^ is a compact, smooth manifold of dimension 3. Hence we need not to 
use the pull-back ^^^(ssi). We can use any section of which is transverse to the zero 

section. 

3.2. Line bundles. In this subsections, we introduce line bundles over Byxr{Pj ct) which are 
defined by families of Dirac operators over y x M and twisted d operators over surfaces in 
y X M. These are used to define a variant of Floer homology groups for 2-torsion instanton 
invariants. 

Let y be a homology 3-sphere and Q = Y x U(2) he the trivial t/(2)-bundle over Y. Fix 
a connection adet on Qdet = Yx f/(l). As before, connections on Q and 7i*Q are compatible 
with Odet, unless explicitly stated otherwise. 

We assume the following. 

Hypothesis 3.3. All fiat connections on Q are non-degenerate. 

For irreducible fiat connections p,a, we define SyxM(P)Cr), Byx^{p,o') as before. We have 
just one gauge equivalence class of projectively fiat connections which are not irreducible. It 
is represented by a connection which induce the trivial connection on the adjoint bundle 
of Q. Fix a smooth map gi : Y ^ SU{2) with degree 1 and put 9a := gi{Oo) for a G Z. We 
define By^^iOayCr), BYxM.{p,da) as usual. Let Fp be the stabilizer of p in Qq. Then we have a 
natural action of F^ x F^. on By^^^ip, o"), and Bp^^{p, a) = By^^^ip, o')/^p x ^a- Note that the 
action of the subgroup {±(1, 1)} C Fp x F^. on By^^{p, a) is trivial. 

For fiat connections p,a, we have real line bundles 

A(p,a) ^ B;.^^{p,cr) 

induced by families of the real part of Dirac operators. Since the action of {±(1,1)} C Fp x F^. 
on the line bundle is not trivial in general, the hne bundle may not descend to Bpy^^{p, a). To 
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avoid this problem, we will introduce real line bundles M(^a,p) and 'R{a,9h) over By^^iPjCr) 
for irreducible flat connections p,a and a,b G Z. Choose connections A{9a,p), A{a,9b) on 
Tr*Q xR such that 

6a on F X (—00, —1) , 
p on y X (1, 00) . 

a on F X (—00, —1) , 



A(ea,p)^ 

\( o\ \ ^ on F X (— 00, — 
^^''^^'>-\9, onyx (1,00). 



Then we have the Dirac operators 

^A^e.,,) : Lf -^^^\S^ ®E)^ Lf-^'^\s- ® E), 

^AiaA) ■■ L'/-^'^\S+ ^E)^ Lf-'''\S- ® E). 

We denote the numerical index of these operators by Ind""*" ^^(ecp); Ind""*" 'PA{a,eb)- Define 
R{9a,p) to be trivial line bundle with action of Vp = Z2 of wight Ind""*" 'PA{ea,p)- Similarly 
M((7, 9i)) is the trivial real line bundle with action of wight Ind""*" '^A{a,eb)- Here we put 

Kb{P: cr) := R(9a, p) ® A(p, a) (g) R((7, 9^). 
The wight of the action of {±(1, 1)} C x T^. is 

Ind-+ '^A(e^,p) + Ind-+ '^a + Ind~+ '^A{aA) = Ind"+ '^Aie^fi,)- 
Here [A\ e ^yx^^Pi'^)- Ind""*" pA{eaA) even, the line bundle Aaft(p, cr) descends to 

Lemma 3.4. Ind"+ '?)A{ea,ei) = -{b - a). 

Proof. Let Pb-a be a f/ (2)-bundle over Y X with ci{Pb-a) = 0, C2{Pb-a) ^ b - a. It follows 
from the additivity of index and the index formula that 

Ind-+ ^Aie.A) = ch(Pb-a)A(Y x S')/[Y x S']. 

Since ch{Pb-a) = 2 - C2{Pb-a) and A(Y x S*^) = 1, we have 

Ind-+ ^Aie.A) = -< C2{Pb-a), [Y x S'] >= -{b - a). 

□ 

For a, 6 e Z with b — a = mod 2, we get a real hue bundle 

Aab{p,(r) := Aab{p,(r)/S0{3) ^ BY^^{p,a). 

Let a, b, a', b' be integers with a = b = a' = b' mod 2. Then we have a natural isomorphism 
from M{9a', 9a)<S>Aab{p, a)<^'R{9b, 9b') to Aa'b'{p, cr) and we can see that this isomorphism induces 
an isomorphism from Aab{p, cr) to Aa'b'{p, cr). Therefore we obtain: 

Lemma 3.5. If a = b = a' = b' mod 2, Aab{p, cr) and Aa'b'ip, cr) are isomorphic to each other. 

Definition 3.6. For irreducible flat connections p,a and a € {0,1}, we write A("''(p, cr) for 
Aaa{p,cr). Moreover we define u^i^ = u'f\p,a) by 

u't^ - ^i(A(«)(p,<7)) e H\B*y^^{p,a);Z2). 
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Let 7 = 5" be a loop in Y. Then we have the determinant hne bundle 

£r ^ i3yxR(p,o-)- 

Here r = 7 X M. This line bundle does not descend to Byxr{p, in general, we can however 
apply the technique used to define Aab{p, cr). 

Let 9j^o be the restriction of Oq to 7. Choose a smooth map : ^ ^ U{2) such that the 
homotopy class [qj] is the generator of 7ri(C/(2)) = Z. Then for a e Z we put 



<(^7,0). 



Note that these connections are not compatible with the fixed connection Odct on Qdct- For 
a,b G Z and irreducible fiat connections p, a, we take connections A(6'^,a, p), A{a, 9^^h) on 7t*Q 
such that 

A{e,.,p)^\^^- on Fx (-00 -1), 
V 'y,a,t-j y p on y X (l,oo), 

A{a,e,,) = i onyx (-00,-1), 
^ ' ^'^ \ 9y^b on y X (1, 00). 

(These connections are not compatible a^et either.) Let C{9^^a, p) and C{a,9^^b) be the trivial 
complex line bundles over ByxRip, with Fp-action of weight Ind""*" dA(e.y,a,p) ro.-action 
of wight Ind""^ dA{a,0j t) respectively. Put 

^r,ab{P^ (^) ■= C(^7,a, p) ® ^r{p, (t) ® C(cT, O^^b) — > Byy^uip, a). 
Then Tp x naturally acts on jC,r,ab{P: and the weight of {±(1,1)} is 

Ind~+ dA(e-,,a,p) + Ind~+ dA(p,a) + Ind~+ dA(a,e^^,) = Ind~+ 
If this number is even, the line bundle Cr,ab{p, c) descends to Byxm.{p, cr)- 
Lemma 3.7. Ind~"+ dA(ej,a,e^,t) ^b-a. 

Proof. Let P^^b-a be a U{2) bundle over 7x5^ with ci{P^fi-a) = b — a. From the additivity 
of index and the index formula, we have 

Ind-+ = chiP,,b-a)Ai^ X ^i)/[7 X S'] 
= {2 + c,{P^,b-a))/bxS'] 
= b — a. 

□ 

This lemma implies that for a, 6 e Z with a = b mod 2 we can define 

^r,ab{p, cr) := Cr,ab{p, cr)/^p xT^^ ^fxmIP' 

We can also show that for a = 6 = a' = 6' mod 2 we have a natural isomorphism be- 
tween CT,ab{p,CF) and CT,a'b'{p,CF)- We get two bundles CP{p,a) := jCr,oo(p,cr), d^\p,a) := 
>Cr,ii(p, cr)- We will use only >Cr^(p, a) to define a variant of Floer homology. 
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3.3. The complex. Using the hne bundles introduced in the previous subsection, we construct 
homology groups which allow us to extend 2-torsion instanton invariants to 4-manifolds with 
boundary. The idea of the construction is fundamentally the same as Fukaya-Floer homology 
groups in Section [2l The main difference is the existence of a reducible fiat connection. 

Fix a G {0, 1} and take sections 

SA = SAip,a) : M°,j,(p,(t) A^''\p,a) 

sc = sc{p,(y) ■■ M°xjj(p,(t) — y C'^\p,a). 

We denote the zero sets by Va, Va- For sets 

Lo = 0, L^ = {A}, L2 = {C}, Ls = {A,C}, 

we define 

M^^^ip, a; Lo) := M^^^{p, a), M^^^{p, a; L,) := M° ^^(p, a) D Va, 
M^^^ip, a; L2) := M^^^{p, a) n Vc, M^^^{p, a; L3) := M^^^{p, a) H H Vc- 

Lemma 3.8. Let p, a be (possibly reducible) flat connections on aU {2)-bundle Q over a homol- 
ogy 3-sphere Y with dim My y;]g^{p,(j) < 3. Take a sequence in Myy^j^iPjCr) converging 
to {[Af], [A^]) with r > 1. Here [A^] E M^^^lp{i - l),p(i)). Then p{i) are irreducible 
for i = 1, . . . ,r — 1. 

Proof. The dimension of dim My xr{p,<^) is given by 

dim My xm(p, a) = lnd++ Da- = Ind"+ Da- - dimi/J, 
where is the Lie algebra of Tp. The additivity of index implies that 

r r 

Ind-^DAc^ = J2^nd-+ Da^ = ^ ( dim MyxR(p(« - 1) , p(i)) + dim H''p^i_,^) . 

i=l i=l 

Hence we have 

r r— 1 

dim MyxR(p, a) = ^ dimMyxiR(p(i - l),p(i)) + ^dimifj(.). 

i=l i=l 

Assume that p{io) is reducible for some io with 1 < io < r — 1. Then p{io) is the trivial fiat 
connection and the dimension of Hp(^i^^) is 3, and dim MyxRipii — l),p(*)) — 1- Hence 

4 > dim My xk(p,o-) > r + 3 > 4. 
This is a contradiction. □ 

This lemma means that we can apply the method in the previous section to showing that we 
can take admissible sections of the line bundles A^°-\p, a), C^\p, cr) when dim My ^[^(p, cr) < 3. 
Thus we obtain the following. 

Lemma 3.9. Let p, a be irreducible flat connections on Q with dim My^]g(p, a) < 3. Then we 
can take sections sa,Sc of A^"'\p,a), C^j^\p,a) with the following properties. 

• Let be a sequence m M^^^{p, a) with [A"] ^ ([A^], . . . , Then we have 

sa([a"]) s^{\Ar]) K ■ ■ ■ K 3A(m)> 

sc{\A^])^sc{[AT])^---msc{[Ar']). 
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• Let (3 be an integer with < (3 <3. // dim My ^jg(p, cr) < (3, then 

• //dim My ^]g(p, 0") = f3, then the intersection My^-^ip, cr; Lj^) is transverse and compact. 
Hence Myy^^^p, cr; Lp) is a finite set. 

In this section, we refer to CFi{Y) as the Z2-vector space generated by gauge equivalence 
classes [p] of irreducible fiat connections with 5y([p]) = i mod 8. Using admissible sections of 
A(")(p, cr), CY\p,<y), we define the complex as follows. 

Definition 3.10. 

(1) 

CFsn-p{Y) \®'L2<0n> if J = 8n, n = a + 1 mod 2 



cf\Y) 



5 



./3=0 



QCFj^p{Y) otherwise. 

/3=0 

(2) We define d^""^ : Cf\Y) C^^\{Y) as follows. 

(a) For integers /3i,/32 with < /3i < < 3 and generators [p] G CFj^p^iY), [a] G 
CFj„/32_i(F), we put 

< d^''H\p\) \a] >■= I #^5^xr(p.^;^/32\^/3i) mod 2 if L^^ C L/^^, 
^'-^^i^''- i ' Y otherwise. 

(b) For [p] G CFsn+i{Y) with n = a + 1 mod 2, we put 

< d^''\[p]),e^ >:= #M°,j,(p,^„) mod 2. 

(c) For [cr] G CFsn-i(Y) with n = a + 1 mod 2, we put 

< d^'^HOn), [cr] >= #M°^K(^„,a) mod 2. 
We claim that d^""^ o c}(") is identically zero. 

Lemma 3.11. d^"^ o d^"^ = 0. 

We must show that for generators [p] G Cj"^(F), [cr] G Cj^^al^) 
(29) < o 9(")([p]), [a] >= mod 2. 

We split the proof into four cases. 

(1) Flat connections p,a are irreducible and [p] G CFj_^^(F), [a] G CFj_^2-2(^) for some 
integers with < - /3i < 2. 

(2) [p] G CF8„_(_2(^)7 d = On for some n G Z with n = a + 1 mod 2. 

(3) p = 6'„, [a] G CF8„_5(F) for some n G Z with n = a + 1 mod 2. 

(4) [p] G CF,(F), [a] G CF,_5(r). 

To prove fl2U]) . we need to describe the ends of My-^^^p^ cr) as in the previous section. In the 
cases (1), (2), (3), the dimension of the moduh space Myj^]g(p, a) is less than 4. Lemma 1X51 
means that fl2I?l) follows from the same discussion in the previous section. Therefore we need 
to consider only the case (4). 
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Lemma 3.12. Take generators [p] G CFj{Y), \a] G CFj^^iY). We can take sections s\, sc of 

the line bundles A(")(p, a), C^\p, cr) with the following property. Let {[A°']}a be a sequence in 
Myxjj(p, cr) converging to some {[A^], . . . , [AJ^]). Here [Af] G My^]g(p(i — l),p{i)). Suppose 
that p{i) are irreducible for i = l,...,r — 1. Then 

sc{\A^])-^sc{\AT])M...msc{\A^]). 

We can show this by using a partition of unity on the moduh space as Proposition I2.8[ The 
following lemma follows from the usual dimension-counting argument. 

Lemma 3.13. Take generators [p] G CFj{Y), [a] G CFj_^{Y). Let {[A°]}q, be a sequence in 
Myxij(p, (t; L3) converging to {[A'^], . . . , [A^]) with r > 1. Then we have either 

• r = 2 and 

[AT] G M°,j,(p,p(l);L^), [A^] G M^,M^),a;L,\Lp) 

for some integer /3 with < /3 < 3 and an irreducible flat connection p(l) with [p(l)] G 
CF,.^_i(r), or 

• r = 2, p(l) = On for some n and 

dimM°^i,(p,ft„) = dimM°^K(^„,a) = 0. 

This lemma means that the reducible flat connections On do not appear in the description of 
the end of My^^^p.cr) if [p] ^ CF^n+iiX) for any n. Therefore fl29l) follows from a discussion 
like that in the previous section in this case. 

From now on, we assume that [p] G CF8„+i(F), [a] G CF^in-iiY). To describe the end of 
the cut-down moduli space Myy^^^p, o\ L^), we put 

3 

N^rr := IJ U xR (P, P(l) J ^Z^) ^ M^y ^^{p{\) , ' L;\L ^) , 

/3=0 [P(l)] 

iV,,rf := M°^k(p,^„) X M°^j,(e„,a), 

iV := Ni„\^Nred. 

Here [p(l)] runs over generators of CF8„+i_/3(F). For each [A] = ([Ai], [A2]) G we have 
a gluing map 

Gl\^ : ?7[Ai] X (To, 00) X f/[^2] — ^ M^^^{p,a) 

for some precompact open neighborhoods U[Ax\-,U\a2\ of [^2] and Tq > 0, and for [A] = 
([Ai], [A2]) G A^red, we have a gluing map 

: (To, 00) X S0(3) ^ M°,K(p,a). 

For Ti > To, we write G'/[a].t>Ti for the restriction G/^a] to the region where T > Ti. Here we 
put 

M' = M^r, := M°,R(p,a;L3) \ |J Im G'/[j,t>t,. 

Lemma 13.131 means that M' is compact. For irreducible flat connections p(l), the sections 
of the line bundles over MyxR(p, p(l)), My^jj(p(l), a) satisfy the transversality conditions in 
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Lemma [3.91 Hence from a discussion like that in the proof of ([7]), we have 

(30) M' n IJ lmGllA],n-l<T<T, = N,rr X (0, 1] 

for large Ti. Perturbing the sections sa, sc outside the end (l30l) of M', we get a 1-dimensional, 
smooth, compact manifold with boundary 

N,rr u U {S0{3)i^nVAnVc). 

Here S'0(3)[a] is the image of SO (3) x {Ti} by G/^a]- We write Sia] for the intersection 
S0{3)[A\ r\V\r\Vc. Then we obtain 



We must show 



(31) #%] = { 

to get f l29|) . This follows from the following 



1 mod 2 if n = a + 1 mod 2, 
mod 2 otherwise 



Lemma 3.14. The restriction Cy\p, cr)\so{3)i^ is non-trivial, and the restriction A^°-\p,a)\so{3) 
is non-trivial if and only if n = a + 1 mod 2. 

Proof. Let Xi,X2 be two copies of F x M. And we write Xi(T), X2(T) for Y x (-oo,2T), 
y X (-2T, oo). Gluing Xi(T), X2(T) through the identification 

Y X (T, 2r) ^ Fx (-2T, -T) 
(l/,t) ^ (2/,t-3T), 

we get a manifold X^^'^K 

Let E' be the rank-two complex vector bundle over Y associated with Q. Take a trivialization 
(p of E such that 6*^ is trivial under 0. We write Ei for 7r*£' and 0j for 7r*0. Here tTj are the 
projections from Xi{T) to Y. For -u G SU{2) we define a vector bundle E{u) by 

Gluing two instantons ([^i], [A2]) € A^red, we get an instanton [A('u)] on E{u), and we have 
the determinant line bundle {^r\A{u)]}u<^su(2) over SU{2). We have an action of {±1} on this 
bundle defined by the following diagram: 

iC^%Aiu)] (C(^,,i,p)®(£)[A,])® ((£)[A,]®C(a,^,,i)) 

(-1)'"1+™2®1 

(£«) lA{-u)] > (^(6*^,1, p)®(£)[A,])® ((/:)[A.]®C(or,0^,l)). 

Here 

mi = Ind"'+ ind9A(e^ m2 = Ind"'+ Ba^- 



FLOER HOMOLOGY FOR 2-TORSION INSTANTON INVARIANTS 



39 



The restriction C^\p,a)\so{3)iAi quotient of {^y\a{u)]}->j-&su{2) by this action. The nec- 

essary and sufficient condition for £p ^(p, o")|5'o(3)[a] be non-trivial is that mi + 1712 is odd. 
From the additivity of the index, we have 

mi+m2 = Ind"'+ dA{e-,,^Mi)- 
Here 6'„|^) is the connection on F = 7 x M with hmits 6'^^i, 6'„|^. Let gi be the fixed 

map from Y to SU{2) with degree 1. The class [5'i|7] represented by the restriction of gi to 7 
is trivial in tti{U{2)). Hence we have 

mi + 1712 = Ind"'+9A{0^,i,e^,o) = -1 
by Lemma [3171 Therefore the restriction C^\p,a)\so{3)iA] non-trivial. 

Similarly the necessary and sufficient condition for A^"-\p, cr)\soi3)iA] be non-trivial is that 

Ind"'+ ^A(e,,e„) = 1 mod 2. 
Lemma 13.41 implies that this condition is equivalent to 

n = a + 1 mod 2. 

□ 

Definition 3.15. &\Y;^) := H,{Ci''\Y),d^''^). 

As in Subsection 12. 6[ we can show the following. 

Proposition 3.16. I^"'\Y;'y) is independent of the metric onY and the sections of A^°'\p, a) , 
4\p,a). 

Remark 3.17. We assumed that F is a homology 3-sphere. In this case, it is sufficient to 
consider a very small loop 7 which represents the trivial class in vri(y) or one point which is 
a degenerate loop, when we calculate invariants using our gluing formula. However we have 
considered a general loop 7 which may not be trivial in 7ri(y). One of the reason is that even 
if we take a degenerate loop, we can see that the components of the boundary operator are not 
trivial, and we need to consider the components as in the case where the homotopy class of 
the loop is non-trivial. Another reason is that we hope our construction is extended to more 
general 3-manifolds. 

3.4. Relative invariants. Let X be a closed, oriented, simply connected, non-spin 4-manifold 
with b'^{X) positive and even. We consider a situation where we have a decomposition X = 
XqUyXi. Here Y is an oriented homology 3-sphere and Xq, Xi are compact, simply connected, 
non-spin 4-manifold with boundary Y, Y and 6"'"(Xo), > 1. We consider invariants 

^xp(-^)' where P is a [/(2)-bundle over X with W2{P) = W2{X) and dim Mp = 3 and z G 
H2{X] Z) with z ■ z = mod 2. The purpose is to write \l/p (z) in terms of data from Xq, Xi 
as in the case of the usual Donaldson invariants. 

We fix a trivialization (po of Q := P\y and we write ^0 for the trivial connection. Choose a 
smooth map gi from Y to SU{2) of degree 1. For n G Z, we denote gi{9o) by 6'„ as before. 
For a sequence 00, put X" = Xq U (F x [0, T"]) U Xi. We can easily show: 

Lemma 3.18. Take [A"] G Mx'^,p and assume that [A"] converges to {[A^], . . . , [^45^]). Here 
[A°°] G My^]g(p(z — l),p(i)). Then we have either 

• p{i) are irreducible for i = l,...,r — 1, or 
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• r = 1 and p(l) = 6n for some n G Z, moreover dimM^^(^„) = dimM^^(^„) = 0. 

This lemma means that the construction of the gluing formula for \E'^^(z) is the same as 
that in Subsection 12.81 if dimM^^(^„) ^ for all n G Z. Hence we suppose the following. 

Hypothesis 3.19. There is an integer hq with dimM^^(^^„,)) = 0. 

We will define relative invariants for Xq under this assumption. Put Pq := P\xo and let 
7 = 5^ be a loop in Y and Sq be an oriented surface embedded in Xq with boundary 7. Using 
Dirac operators associated with a spin-c structure cq over Xq with ci(detco) = — ci(Po) and 
twisted d operators over we have line bundles 

A(p) Ai5i,,(p), 4„^%(p)- 

For irreducible flat connections p and a G {0, 1} we put 

A^-^) := A(p) ® l(p,^,), £g(p) := 4,,(p) ®C{p,e,,,). 

Here 6^^i is defined as the previous subsection. Then Tp = Z2 naturally acts on these bundles. 

Lemma 3.20. Let Lpo be the fixed trivialization of Q and put ipa := Qi^Po- When C2{Po, ^a) = 
mod 2, A^'^\p) descends to B*^^^{p). Let zq G H2{Xo, Y; Z) = H2{Xq] Z) be the class represented 

by Sq. If Zq ■ Zq = 1 mod 2, the line bundle jC^^\p) also descends from B*^^ (p) to B*^ (p). 

Proof. The wight of the Tp action on A*^"^(p) is Ind^ 1^A(0a)- Hence it is sufficient to show that 
the index is even if C2{Po\ fa) is even. Choose a compact, spin 4-manifold Xq with boundary 
Y and we consider a manifold X' = Xq Uy Xq. Let P' be a [/(2)-bundle over X' obtained by 
gluing Pq and the trivial bundle Px^ over Xq through ipa- Let Cg be a spin-c structure over Xg 
induced by a spin structure. Then we have a spin-c structure c' on X' induced by cq, Cq and 
we have the Dirac operator 

^A' ■■ T{S^ ® E') T{S^ ® E'). 

Here A' is a connection induced by A{6a) and the trivial connection 6x1^ on and E' is the 
complex vector bundle of rank two associated with P'. Since , is the direct sum of two 

copies of a Dirac operator, the index Ind~ Vq , is even. Thus we get 

^0 

Ind+ '^A{e,) = Ind+ ^A(e,) + Ind~ ^e^, mod 2 
= Ind '^A' mod 2. 
Applying the index formula, we have 

Ind^A' = c/i(E')i(X')e5^i('i<='^')/[X'] 

= 2i(X')e^^^('i^*'=')/[X'] + ici(P')(ci(det c') + c^{E'))/[X'] - C2{E')/[X']. 

Since i(X')e5^i('^<=*^'V[X'] is the index of a spin-c Dirac operator, it is an integer. Hence the 
first term is even. From ci(detc') = ~ci{E'), the second term is zero. Therefore we obtain 

Ind+ ^Aie.) = -C2{E')/[X'] = C2{Po; fa) mod 2. 

Thus the first part of Lemma follows. 

To show the second part, we need to show 

Ind+ dA(9^^,) = mod 2. 
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Here A{e^^ i) is a connection on Xq with hmit 9^ i. The additivity of index imphes that 

Ind+ = Ind+ <9a(9^,o) + Ind"+ <9A(e^,o,e^,i)- 

Applying Lemma 13.71 to the second term in the right hand side, we get 

Ind+ <9A(e^,i) = Ind+ dA{e^,o) + 1- 

Let ip^ be the restriction of ipo to 7 and take an oriented, compact surface Sq with boundary 
7. Gluing PqIso and the trivial [/(2)-bundle Ps(, over Eg through ip^, we get a f/(2)-bundle P^/ 
over S' = So Sq. Let 6'' be the trivial connection on Pq. The index Ind~ de' is even, since 
dgr is the direct sum of two copies of the 5-operator. Thus we have 

Ind+ dA(e^^o) = Ind+ <9A(e^,o) + Ind" Be' mod 2 
= Ind(9yi/ mod 2. 

Here A' is the connection over E' induced by A{9^ q) and 9'. From the fact that ci(Po) = 'U^2(-^o) 
mod 2 and ■ 2:0 = 1 mod 2, we have 

Ind^A' = c/i(Ps')^(S')/P'] 
= (2 + ci(PsO)/[Sl 
=<Ci(PsO,[S'] > 

= < Ci(Po),2o > 

= 1 mod 2. 



Therefore we obtain 



Ind+9A(9^ j) = mod 2. 

□ 



Definition 3.21. Let a G {0, 1} with C2(Po; v^a) = mod 2. For irreducible flat connections 
p, we define 

A('^)(p) := A('^)(p)/r, ^ B^^ip), £g(p) := Zg/F, A B*^^{p). 
The following lemma follows from a dimension counting argument like the proof of Lemma 

m 

Lemma 3.22. Let p be a flat connection on Q with dim M^^(p) < 4 and {[A°']}a be a sequence 
in Mj^^{p) which converges to some ([^4^^], . . . , [AJ^]). Here [A°°] G My^^ip^i — l),p(z)) and 
p{r) = p. Then p{i) are irreducible for z = 0, . . . , r — 1. 



This lemma implies that for p with dim M^^^(p) < 4 we can take sections s\{p) : Mj^^^{p) — )• 



A*^")(p), sc{p) '■ Mj^ (p) —7- C[~^\p) having properties like those in Proposition 12. 8[ Using 



admissible sections, we define ijj'^^ G Cg'^^{Y) as follows. Let /3 be an integer with < /3 < 3 
and take a generator [p] G CPgno-^sl^)- Then put 

<rx\,[p]>--=mxoip-^Lp) mod 2, 

and we define 

n ^ _ / ^^Xo(^o) mod 2 if no = a + 1 mod 2, 
<Vxo^'^no>- I Q otherwise. 

These numbers define a chain ip'^^ G Cg'^^{Y) as usual. As before, we have: 
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Proposition 3.23. The chain if)"^^^ is a cycle. Moreover the class = ["^Axo] ^ ^snoO^'^l) ^■^ 
independent of the metric on Xq and the admissible sections of A.^°'\p), C^^{p). 

Remark 3.24. As the case of relative Donaldson invariants ^ Xoi if ^in analogy of Conjec- 
tur d2.25l holds then relative 2-torsion instanton invariants '^^x^ is independent of the surface 
So representing the homology class zq. 

3.5. Gluing formula. We begin with the definition of a pairing li''\Y)®li''\Y) ^Z2. Since 
5y([p]) = — 5y([p]) — 3 mod 8, we have a natural pairing CFj{Y) ® CF_j_3(F) — t- Z2. For 
[6n\ G C'^^iY) and [6'_„] G C^^l^iY), we define < 6n,0_n >= 1 mod 2. Then we get a pairing 

We can easily show 

Lemma 3.25. For [p] G and [a] G 

<d^^\[p]),a>=< [p],a(")([a])>. 
This lemma means that the pairing induces a pairing 

The main statement in this section is the following. 

Theorem 3.26. Let X be a closed, oriented, simply connected ^-^o-nifold with positive 
and even. Assume that X has a decomposition X = Xq Uy Xi, where Y is a homology 3- 
sphere and Xq,Xi are compact, non-spin, simply connected ^-^o-nifolds with boundary Y , Y . 
Take a loop jinY and compact, oriented surfaces Eg, Ei in Xq, Xi with boundary 7. // the 
self-intersection numbers of Zt := [Ej] G H2{Xi,Y;Z) = H2{Xi;Z) are odd, then we have 

vl/"^^) =< n^,(Eo),n;(EO >, 

where z = Zq -\- Zi E H2{X; Z). 

We can generalize this formula to the case when the dimension of Mx,p is more than 3. 
Assume that the dimension of Mx,p is 2((io + c?i) + 3 for some do,di G Z>o. For Xi, . . . , XdQ G 
H2{Xq; Z), yi, . . . ,yd^ G H2{Xi; Z) with Xt ■ Xi = yj ■ yj = mod 2, we can define relative 
invariants 

. . . , Eo) G li^\Y; 7), . . . , Ei) G li^\Y; 7). 

Here Ej are surfaces in Xi as in Theorem 13.261 Then we have 

=< ^'^^^(xi,...,Xdo,Eo),^'^\(?/i,...,l/di,Ei) > . 

Applying this formula to the case Y = S^, we immediately obtain the following. 

Corollary 3.27. Let Xq, Xi be closed, oriented, non-spin, simply connected 4-i^o,nifolds with 
positive and odd. Let Pi be U{2)-bundles over Xi with W2{Pi) = W2{Xi), C2{Pi) = 1 
mod 2 and dim Mp. = 2di for some di > 0. For homology classes xi, . . . , Xd^, zq G H2{Xq] Z), 
1/1, ... , ydj^, Zi G H2{Xi; Z) with Xi ■ Xi = yj ■ yj = mod 2, Zq ■ Zq = Zi ■ Zi = 1 mod 2, we 
have 

^^^(xi, ...,Xdo,yu...,ydi,z) = ^XoM^i^ ^Xi,Pi(z/i, • • • , mod 2. 

Here z = zq + zi E H2{X] Z). 
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Remark 3.28. In general, usual Donaldson invariants ^'p(a;i, . . . ,Xd) do not lie in Z even if 
Xi G H2{X;Z). However if < W2{P),Xi >= mod 2 for all i then the value \l/p(xi, . . . ,Xd) 
lies in Z. Therefore the above formula makes sense. 

Befoer we show Theorem 13.261 we prove non-vanising results using Corollary 13.271 

Theorem 3.29. Fori = 0,1, let CFj and CP,^ be copies of CF^ andCF'^ respectively. Let 
Hi G H2{CFj; Z), Ei E H2{CF'^] Z) be the standard generators. Then we have 

^^;g#„?(^o + i/i) = l mod 2, 

*CPg#CF?#CPg iHo + Eo,Ho + H,)^l mod 2, 

KU^r.2^rW2^r^^iHo + Eo, H, + E,, H, + H,)^l mod 2. 



In [121 Proposition 7.1], Kotschick showed \l/cp2,p = ~1 for the f/(2)-bundle P with ci = if, 
C2 = 1. (In this case, the dimension of the moduli space is zero.) Hence the first equality 
in Theorem 13.291 follows from Corollary 13.271 We also have ^cp2#cp2,p(-^ + E) = 1 for the 
t/(2)-bundle P with ci = ii - ca = 1 by m Proposition 7.1]. (Note that in [12], 2/i([E]) is 
used to define the invariant.) Hence we obtain the second and third equations from Corollary 
13:271 

We begin the proof of Theorem 13.261 As mentioned before, if there are no integers n 
such that dimAf^^(5„) = then the proof of Theorem 13.261 is quite the same as that of 
Theorem 12.291 Hence we suppose Hypothesis 13.191 To prove Theorem I3.26[ we fix a sequence 
— )• 00 and consider manifolds X" = Xq U (F x [0,T"]) U Xi as usual. As in Subsection 
12. 3[ we can take sections : Mx^' — ?• A, : Mx'^ Cj^ with the following property. 
Suppose that a sequence [A"] e M^- converges to {[A^], . . . , [A~]). Here [A^] G M^y(p(0)), 
[Af] e M^xig(p(z - l),p{i)) (z = 1, . . . ,r - 1), [A^] G M^^{p{r - 1)). Moreover suppose that 
p{i) are irreducible for i = 0, . . . , r — 1. Then we have 

sl{\A^])-^s^{[A^])m-..ms^(\A^]), 

sl{\A'^])^sc{\A^])^...Msc{\A^]). 

A standard dimension-counting argument shows that for any sequence [y4°] G Mx-^iL^) = 
Mx" n Va n Vc, there is a subsequence {[A"']}^' such that [A"'] converges to ([^[f], [^?°]), 
where [A^] G M^^(p;L^), [A^] G M ^^{p; L^\L p) for some < /3 < 3 and [p] G CF8„„_^(r), 
or [A^] G Mj^^{9no), [A'^] G M^^(^„o). Hence for large a we have 

Mx.(L3) = |J U M^^(p;L;3)xM^^(p;L3\L^)U |J ^0(3)[^ n I^a n F^. 

Here 5*0 (3) is the image of SO {3) by a gluing map as in the proof of Lemma [3. Hi Therefore 

we get 

n (^) = E E *^Xo (P; Lp) ■ #M^^ (p; L3\L;3) + ^ # (^^(3) n V^A n V^) . 

/3 [p] [A] 

Thus we must show 

1 if no = a + 1 mod 2, 



This follows from the following lemma. 
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Lemma 3.30. For each [A\ = {[Aq], [Ai]), the restriction £e|50(3)[^ is non-trivial. And the 
restriction A\so{3)i^ is non-trivial if and only if Uq = a -\- 1 mod 2. 

Proof. As in the proof of Lemma I3.14[ we can see that a necessary and sufficient condition for 
the restrictions of the fine bundles to be non-trivial is that the indexes Ind"^ dA_o , Ind"^ "^Aq 
odd. The calculations in the proof of Lemma 13.201 show that 

Ind"*" = zq ■ Zq = 1 mod 2 

Ind"'' = no — a mod 2. 

Therefore we obtain the statements. □ 

4. A NON-EXISTENCE RESULT 

Let X be a closed, oriented, simply connected 4-manifold and Qx be the intersection form 
of X on H2{X; Z). For positive integers k and [Si], . . . , [T,2k] ^ H2{X\ Z), we define 

Q%\^l], [Safe]) := ^ Qx{[^a(l)], Pa(2)]) ■ ■ •<5x([S^(2fc-l)], [^ai2k)])- 

■ <TG62fe 

In [S], Donaldson showed that when X is spin and b'^ = 1, Qx must satisfy the condition 

(32) Qg)([Si],...,[S4]) = mod 2 

for all [Si],...,[S4] eH2{X;Z). 

In this section, we will consider the intersection forms of compact 4-manifolds with bound- 
aries homology 3-spheres. We prove that the condition also holds for compact, spin 4-manifold 
X with = 1 and with boundary a homology 3-sphere Y, if HFi{Y; Z2) and HF2{Y; Z2) are 
trivial. Here HF^,{Y\ Z2) are the usual Floer homology groups with coefficients Z2. 

Theorem 4.1. Let Y he an oriented homology 3-sphere with HFi{Y; Z2) = 0, HF2{Y; Z2) = 0. 
IfY bounds a compact, spin, simply connected 4-''nanifold X with b^{X) = 1, then we have 

Qg)([Si],...,[S4]) = mod 2 
for any [Si], . . . , [SJ G //2(X; Z). 

This is originally due to Fukaya, Furuta and Ohta. (See [IZ].) Making use of techniques 
developed in this paper, we write down the proof of this theorem. 

4.1. Relative invariants for spin 4-manifoIds. In the previous section, we defined relative 
invariants for non-spin 4-manifolds with boundary. Here we define relative 2-torsion instanton 
invariants for spin 4-manifolds, which are used to prove Theorem 14. 1[ To define the invariants, 
we modify the construction in the previous section. 

Let Q = Y X SU{2) be the trivial S'?7(2)-bundle over Y and fix a G {0, 1}. For irreducible 
flat connections p, cr on Q with dim My (p, cr) < 2, we can take sections sa : Myxjj(p, a) — )• 
A*^°^ (p, cr) which are compatible with gluing maps and satisfy the transversality conditions. 

Definition 4.2. Put Cj{Y) := CFj{Y) © CFj_i{Y). We define 9^'^) : Cj{Y) Cj^i{Y) as 
follows. Let /3i,/32 be integers with < /3i < /32 < 1 and take generators [p] G CFj_j3g, 
[a] G CF,_^,_i(F). If P2-Pi = 0, put 

< d^^\[p]), [a] >:= j^M'y^^{p,a) mod 2. 
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If ^2-Pl = 1, put 

< d^^'Wp]), [a] >:= #(M°,^(p,cr) H Va) mod 2. 

We define d^'^^ by using these numbers as usual. 

Let {[A"]}^, be a sequence in My^^^p, a) for some fiat connections p, cr with dim My^^ip, o") < 
2. We can show that if [A°'] converges to some {[A^], . . . ,[A'^]) e M^^^{p, p{l)) x ■■■ x 
My^]fj(p(r — 1), cr) then p{i) are irreducible for z = 1, . . . , r — 1. This means that we can prove 

as in Section [21 

Definition 4.3. &\y) := H,{C,{Y),d^"'^). 

Consider a compact, simply connected, spin 4-manifold X with boundary Y and h^{X) = 1. 
We will define a relative invariant 

Take a 5't/(2)-bundle P over X and a trivialization ip of P|y with 

C2(P; v^) = 2 G F; Z) = Z. 

We denote by 6 the trivial connection on P|y with respect to y?. The dimension of the moduli 
space Mj^{6) of instantons on X with limit 6 is 

8c2(P; - 3(1 + b+{X)) = 8 • 2 - 3(1 + 1) = 10. 

For generators [p] G CP^(y), we have moduli spaces Mj^{p) of dimension 10 — f3. 

Let E be a closed, oriented surface embedded in X. We put B^_^_ = U {[On]}- It is 
known that the determinant line bundle £s — ^ B^ extends to -B^ (See [9].) We use the same 
notation £s for the extension. We can take a section : _,_ — )■ £e such that ss does not 
vanish near [6*2] . We always assume that sections of £s have this property. Let be the zero 
locus of ss and we define M-^{p) fl Vs to be 

{[A]eM^{p)\s^{[A\^]) = Q}. 

Take homology classes [Si], . . . , [S4] G H2{X;Z). For generic surfaces Si,...,S4, the 
intersection of any two surfaces is transverse and the intersection of any three surfaces is 
empty. Moreover for generic sections s^^, . . . , ss^, all generators [p] G CF^iY) and all subsets 
{li, . . . ,ls} C {1, . . . , 4}, the intersections 

s 

M^(p;S,„...,Sj = M^(p)n f|Ks,^ 

m=l 

are transverse. 

A standard dimension-counting argument shows the following: 

Lemma 4.4. Let p be aflat connection on Q. Assume that a sequence {[A°']}c, in Mj^{p; Si, . . . , S4) 
converges to ii[A^],Z,),...,{[Ar],Zr)). Here {[A^], Z^) G M^(p(0)) xSym"«(X) andi[Ar],Z,) G 
(MyxM(p(^-l),p(^)) X Sym"'(F xR))/M /or z = l,...,r. 

(1) If [p] G CF2{Y) and dimM^(p) = 8 then r = 0, = 0. 

(2) If [p] G CFi{Y) and dim M^(p) = 9 then r < 1 and all Zi are empty. Moreover when 
r = 1, p(0) is irreducible. 
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(3) If p = 6, then one of the following holds: 

• r = 0, Zo = 0. 

• r = and Zq = {xi,X2} for some Xi G S;^ fl S;^, X2 G S/3 fl S/^ with /i, ... 5/4 
distinct. 

• r = 1, p{0) is irreducible and all Zi are empty. 

• r = 2, p(0),p(l) are irreducible and all Z^ are empty. 

It follows from the lemma that we may take admissible sections sa(p) : Mj^{p; Si, ... , S4) — j- 
A(p) for [p] G CFi{Y). We define a chain = ^^^(Si, . . . , S4) G Cf^(F) as follows. For 
[p] G CF2(F) put 

< [P] >■■= #(^x(P; Si- • - S4) n Vj,) mod 2. 
For [p] G CFi(r), put 

< rx\ [p] >■= ^Mjiip; Si, ... , S4) mod 2. 
These define a chain ■^/'^^ G C2°^(F) as usual. 

Lemma 4.5. T/ie cham G Cf^(F) zs a cycle. Moreover the class ^^^([Si], . . . , [S4]) = 

[V^x] ^ -^2''''(^) independent of the metric and the sections and depends only on the homology 
classes [Si], . . . , [S4] G H2{X; Z). 

4.2. Proof of Theorem 14.11 We start with the following lemma. 

Lemma 4.6. If both of HFj^i{Y;Z2) and HFj{Y;'L2) are trivial, then I^"''^ {Y) is also trivial. 

We can easily show this by considering the long exact sequence associated with a short exact 
sequence 

CF,^i{Y; Z2) Ci''\Y) ^ CF,{Y; Z2) 0. 
Next we define a map 

^(a) . j{^a)^Y) ^ Z2. 

For [p] G CFi(r), put 

D('^)([p]):=#MO,Jp,^) mod 2 

and for [p] G CF2(r), put 

D('^)([p]) := #(MO,j,(p,^) n Va) mod 2. 

Then we have a linear map D'-'^^ from C2"'\y) to Z2. Counting the number of the ends of 
appropriate 1-dimensional moduli spaces, we get 

Hence we obtain 

]j{a) . j{a)^Y) — ^ Z2. 

(We denote the map by the same notation.) 

Theorem 14.11 immediately follows from Lemma 14.61 and the following proposition. 

Proposition 4.7. The image of ^^'(Pi], . . . , P4]) G ^(F) by D^o) zs 

g2^([Si],...,[S4]) mod 2. 



FLOER HOMOLOGY FOR 2-TORSION INSTANTON INVARIANTS 



47 



To prove this, we consider the ends of the cut-down moduh space M-^{6; Ei, . . . , E4) n Va 
with dimension 1. Let {[A"]}^ be a sequence in the cut-down moduh space. Then there is 
a subsequence {[A^ ]}„/ which goes to (([^0], -^o), • • • , ([^5^]; Zr)). We can see that we are in 
one of the following cases: 

(1) r = 0, Zo = 0. 

(2) r = 0, = {xi, X2} for some Xi G S;^ fl X2 E fl with /i, . . . , ^4 distinct and 
[A^] = [Ox]- 

(3) r = 1, all Z, are empty, and [p(0)] e CF^iY), [A^] e M^(p(0); Si, . . . , S4), [Af^] e 
M^xR(p(0),^)nVA. 

(4) r = 1, [p(0)] G CF,{Y), [A^] G M^(p(0); E^, . . . , S4) H Vj,, [A^] G , Jp(0), 

We describe the ends of the moduli space relevant to the case ([2]). Let xi G S;^ fl S^^, 
X2 G n for distinct li, . . . ,1^ and take small neighborhoods f/^;^, There is an 

5'0(3)-equivariant map 

k:U^^x U^^ X S0{3) X S0{3) x (Tq, 00) x (Tq, 00) ^ = H+{X)®5u{2) 
such that /t~^(0)/S'O(3) is homeomorphic to an open set in Mj^{6). As in [5] we can see that 

{K-\o)/so{3))nVj,,n---nVj,, = {xj x {xs} x£x {i} x { (t,t) | r>To }, 

where £ is a loop in 5*0(3) which represents the generator of 711(5*0(3)) = Z2. For each 
x = (xi, X2), we have a gluing map 

Gl,:ix{{T,T)\T>To}^ M^(^; Si, ... , S4) 

which is given by gluing the trivial connection on X and two copies of the fundamental in- 
stanton J over 5*^ at xi,X2. The intersection of the image of this map and Va is one of the 
ends of the moduli space. 

Let [A] = {[Ao],[Ai]) be an element of M^^ (p; Si, . . . , S4) x [M^^^{p,e) n Va) for [p] G 
CF2{Y), or an element of (M^(p; Si, . . . , S4) n 14) x M^^^{p, 9) for [p] G CF^{Y). Then we 
have a gluing map 

GliA] ■■ (To,oo) ^M^(^;Si,...,S4). 

The ends of the moduli space relevant to the case ([3]), (jl]) are the intersections of the images 
of these gluing maps and Va- 
Put 

M' = M^(^;Si,...,S4) \ (|JlmG/^u|JlmG/[^). 

X [A] 

If we perturb the section sa over a compact set in M' then the intersection M' fl Va is a smooth 
compact manifold with boundary 

U M^(p; Si, ... , S4) X (M° ,^(p, 6) n Va) 
[p] 

U U (M^(a; Si, ... , S4) n Va) x M^^^{a, 9) 

u|jGy£,To)nVA. 
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Here [p] runs over the generators of CF2{Y) and [cr] runs over the generators of CFi{Y). 
Therefore we get 

. . . , [S4])) + 5^#(Gy^,To) n V^a) = mod 2. 

X 

We can see that 

^Gl^{i,To)nV^) = l mod 2 

for each x. This follows from arguments similar to those which deduce (!3T|) . The point is that 
J is an instanton on an S'f/(2)-bundle P' with C2(P') = 1 and hence the index IndDj is odd. 
Therefore we get 

D(°)(n^([Ei], . . . , [E4])) = 5^ 1 ^ . . . , p4]) mod 2 

as required. 
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